AI infrastructure / GPU electricity
GPU Server Power Consumption: Idle, Load and Annual Cost
A GPU utilization percentage is not a power percentage. Estimate yearly cost from measured server watts and actual time spent in each operating state.
Sections in this article
A procurement spreadsheet lists an eight-GPU server at 9 kW maximum power. The finance team multiplies 9 × 8,760 hours and calls the result its annual electricity bill. That is a useful upper-bound scenario—not a forecast of how the server will actually operate. The same machine can spend hours ingesting data, serving uneven inference traffic or waiting for a queue.
Practical answer: estimate consumption from measured whole-server idle and active draw, time in each state, the number of machines, and the applicable electricity price. Use PUE only when you intend to allocate a consistent share of facility-wide overhead. Peak ratings remain important for circuit design, but they are not a proxy for annual kWh.
GPU server electricity cost calculator
Enter illustrative or measured power at the AC input of one entire server—not per GPU. The active share means fraction of time at the specified high-load draw, not the GPU-utilization counter. The remaining operating hours are assigned to idle. If machines are shut down part of the year, model that separately rather than calling shutdown hours “idle.”
Illustrative duty-cycle estimate. The applied PUE allocates more than just cooling power.
What exactly counts as idle or active power?
The host is powered on and available but not performing the modeled active workload. Memory, NICs, CPUs, control electronics and some fans can still consume substantial power.
The host is executing a representative training or inference phase. Its power must include every accelerator plus CPU, memory, networking and cooling devices inside the server.
A powered-off server should not automatically be counted as idle. Add a separate state if fleet shutdown or standby consumes meaningful annual hours.
For electricity billing, meter the server AC feed or rack PDU where possible. NVIDIA DCGM exposes GPU board power and, on supported devices, energy counters; those readings can reveal phase changes, but a sum of GPU board values excludes other server components. Sampling also smooths rapid transitions. Label the sampling interval and keep measurement locations consistent.
An 85% SM-activity reading should not be interpreted as “85% of maximum server wattage.” Different computational kernels, interconnect activity, memory access and power policies produce different watts even at similar utilization readings. Build a short reference profile with representative jobs and use your observed time-weighted averages.
Worked example: a 100-server training and inference fleet
Suppose measured whole-node power is 1.1 kW while ready and 7.2 kW during the chosen active job. One hundred servers are energized year-round and spend 55% of hours in that active state. These are hypothetical inputs, not a published benchmark for any specific accelerator.
Annual fleet IT energy = 4.455 kW × 100 servers × 8,760 h = 3,902,580 kWh. At PUE 1.25 and $0.12/kWh, allocated facility electricity cost is approximately $585,387 per year.
The calculator shows how changing just the active share from 55% to 75% alters the electricity estimate without changing the machine or the breaker. Higher utilization can raise the bill while producing more useful training work. That is why electricity per completed job is often a better optimization metric than minimum instantaneous watts.
Where this calculation stops being reliable
| Boundary | What the number tells you | What it cannot tell you |
|---|---|---|
| GPU board telemetry | Approximate accelerator draw and trends | Whole-server energy at the rack outlet |
| Rack PDU metering | Real input energy to the supplied IT load | Which process caused that energy |
| PUE allocation | Indicative share of facility energy under matching boundaries | Marginal cooling expense for one additional rack |
| Full-load nameplate | Starting point for validating worst-case design demand | Expected yearly duty cycle or energy |
The Green Grid defines PUE as total facility energy divided by IT equipment energy. PUE includes power distribution, cooling and other overhead; it is not a “cooling multiplier.” A facility-wide annual PUE applied to an individual server is a planning allocation, not a measured marginal cost. Verify that base IT power and the PUE measurement interval cover comparable periods.
Power caps, CPU limits and job scheduling change energy use as well as runtime. If throttling reduces watts but extends a job, total kilowatt-hours may rise, fall or remain nearly unchanged. Test with joules per completed useful workload before declaring savings.
Five checks before sending the estimate to finance
- Capture rack-PDU energy for a representative week or operating cycle, not only the GPU board limit.
- Segment telemetry by workload class: training, inference, warm standby and genuinely powered off.
- Use contracted electricity rates and disclose demand charges or time-of-use pricing not captured here.
- Confirm whether PUE reflects the same site and period. Where possible, report IT and non-IT electricity separately.
- Run low, central and high duty-cycle scenarios; label the result as an estimate until calibrated against invoices.
The useful output is not a single reassuring number. It is a documented relationship between server operating states, energy and billable cost. Retain the source measurements so that the next accelerator generation can be budgeted with comparable assumptions.
How would more accurate billing change this estimate?
The tool deliberately uses one blended electricity rate. A production budget may also need contracted demand charges, power-factor penalties, peak-hour energy rates and the billable boundary between customer and operator. Those charges do not scale directly with kWh, so they must be modeled separately. If your fleet can shut down between projects, measure the time and electricity used by startup, warm-up and job loading before adding a third state. Publish an uncertainty range rather than hiding those conditions in a single annual cost.
Sources and further reading
Primary frameworks and reference materials. Engineering decisions require site-specific validation; legal classifications require assessment of the actual use case.
- GPU power and energy field identifiersNVIDIA Data Center GPU Manager
- Profiling GPU workload utilizationNVIDIA DCGM
- PUE measurement boundaries and interpretationThe Green Grid / Lawrence Berkeley National Laboratory