I was looking through the Yotta agenda and one of the panels stuck out to me: "The Metrics That Matter: How Should We Measure Success in the AI Infrastructure Era?". This panel points to the reality that in some respects we’ve put the cart before the horse when it comes to AI. We know AI is rapidly evolving (perhaps faster than we all expected), we know we need a massive infrastructure buildout, but in all of this what are the north star metrics that we point to? What is success defined as in this massive buildout?
What’s Wrong with Traditional Metrics?
Many folks have called out PUE for being an outdated metric. I agree that while it is still useful to measure the building efficiency it lacks the full context for a north star metric. Using PUE as a north star ignores the entire white space and assumes we have no control or useful input over this. While there certainly are contractual limitations in certain situations, we should not ignore the reality that the goal of data centers and the power behind them is to produce compute that then enables a result to come about for millions of businesses and end users.
Jensen Huang refers to the value chain to produce that end result as a 5-layer cake and it’s unrealistic to assume that a single enterprise could deliver the full cake. However, it’s crucial that we acknowledge the full cake and understand how each layer can impact the others. For this reason our north star is Compute per MW which addresses the energy, chips, infrastructure, and workloads all required to deliver the output.
PUE Measures the Building, AI Infrastructure Needs a Metric That Measures the Business
Compute Per MW
At its core, Compute per Megawatt (MW) shifts the focus from simple facility efficiency to operational yield. Standard facility metrics like PUE treat everything above the physical data center layer as an unchangeable black box. Compute per MW, by contrast, bridges these layers. It recognizes that true efficiency is a co-design problem requiring alignment from electrons to algorithms.
So how do we measure it? In its simplest form this is the formula for Compute Per MW:
But both the numerator and denominator are tricky numbers with a lot of nuance. Different types of data centers will have different measurements for each. The simplest example is a token factory for inference, its unit of delivered throughput is tokens and if we assume that the whole facility is owned/operated/managed by a single company we can divide tokens by the total facility power. The resultant north star metric is tokens/sec per MW. Importantly this metric is not just a % which sometimes fails to capture the gravity of the situation. “1.4 MW available for compute” reads a lot differently than “20% more efficient”.
So how do we measure success for datacenters that are not token factories?
Colocation: the Gap Between Power Contracted and Power Used
A colo sells committed capacity. Revenue is booked per kW under contract, which means Revenue per MW is largely a leasing question until you look at what's actually happening in the building.
Then the gap shows up. Tenants contract for more than they draw. That diversity has always existed, and the standard response is at best cautious oversubscription but in reality utilization still hovers around 50% at best. So the megawatts sit there. Contracted but undrawn, unsellable because you can't see the load well enough to underwrite the risk. The risk aversion is understandable because stretching too far risks an outage, while being conservative just costs you margin. But risk aversion and risk management aren't the same thing and visibility into actual load characteristics enables intelligent risk management.
Colocation facilities could significantly bring visibility to this problem and maximize compute per MW (and revenue per MW) if they enabled a single source of truth for telemetry across OT and IT and layered in capacity intelligence.
Neocloud: GPU-hours per MW
A neocloud doesn't sell megawatts, it sells GPU contracts or GPU-hours. Decompose Revenue per MW and you get sellable GPU-hours per MW multiplied by realized rate. In the same way the colocation facility doesn’t control the IT load, the Neocloud doesn’t control the workload, their tenants do. However, they have far more insight into the IT side given they do all of the cluster management. So how can a neocloud maximize compute per MW?
We really need to look at it from a power utilization AND a GPU utilization angle:
Neoclouds can benefit from the same type of intelligent capacity planning, and in addition they also have the luxury of deploying novel solutions to increase GPU utilization ranging from multi-instance GPUs to serverless compute.
Let the North Star Guide You
A north star metric isn't a scorecard. Its real job is to help make decisions day to day with it as the guiding principle. Think about the following decisions and how the framing of maximizing compute per MW might alter it:
Purchasing decisions - deciding what systems to invest in and the hardware and software choices that will maximize compute per MW. Without this metric you may optimize for cost alone based on nameplate ratings, but with this metric you may also optimize for ability to integrate intelligence into the equipment or the interoperability of hardware and software.
Instrumentation - deciding what and how to measure. This is where most operators actually are today. Almost nobody can compute this number right now, and not because the concept is difficult. The inputs live in four systems that don't talk to each other: device telemetry, IT systems, the facility BMS, and billing. It brings up the old adage “you can’t improve what you don’t measure” and now seems like a great time to start measuring!
Operations - taking actions in real time. The status quo of operations today seems to be set it and forget it to the detriment of the metrics. This is largely because operations teams are incented only on reliability. If we expanded that incentive picture to maximization of output (like a traditional manufacturing facility or chemical plant) you would see that most operators would adopt dynamic operations.
Now back to the Yotta panel, if I were on it this would be my closing message: Success in this buildout isn’t the cheapest megawatt or the lowest PUE. It's how much useful work we pull out of every megawatt the grid was willing to give us, at a moment when the grid is the scarcest thing in the equation. At PADO, we are here to extract all useful compute out of every MW the grid has to offer which is why we created a platform that can be deployed at existing data centers or leveraged to speed up development of new data centers. Want to see how much you are leaving on the table? Book a free site consultation below.
