Compute operations / Scheduling
AI Workload Scheduling: Power and Cooling Constraints
A scheduler can find an available GPU and still choose the wrong place to run it. The missing variable is often the facility capacity behind that server.
Sections in this article
Placing AI jobs is no longer just a question of free GPUs. A training job may be compatible with a node while the associated power branch is close to a tested limit or the coolant distribution unit has little spare capacity. In a multi-tenant facility, the compute scheduler may not even have direct access to the provider's control systems.
Schedule against verified power and cooling envelopes—not a single site-wide utilization number.
Which constraints should inform placement?
A scheduling policy should combine job requirements with hard engineering constraints. Not every constraint is relevant to every architecture: an air-cooled room, direct-to-chip deployment and cloud region require different inputs. Values must be collected at the correct physical boundary rather than inferred from an average facility PUE.
| Signal | What it tells the scheduler | What to verify |
|---|---|---|
| Rack / bus power headroom | Whether another GPU workload fits the actual electrical allocation | Breaker, PDU, branch, diversity and reserved headroom |
| Available cooling duty | Whether the zone can remove additional heat | Supply temperature, flow, CDU limits and degraded redundancy |
| Job characteristics | What resources and duration the work requires | Checkpointing, priority, data locality and interconnect topology |
| Telemetry quality | Whether the decision is based on current conditions | Timestamp age, missing sensors and measurement location |
GPU jobs also differ in portability. A distributed training run may be difficult to move between locations because of synchronization and network costs. A batch inference job with loose latency requirements may be more flexible. Scheduling logic needs to model both, rather than assuming all work can be relocated without consequence.
Worked example: selecting a zone for a new job
Consider a hypothetical 240 kW training workload. Three zones have spare IT power, but their cooling allocations differ. The figures are illustrative, assume the power and cooling figures use a consistent IT-load boundary, and are not facility specifications.
| Zone | Electrical headroom | Cooling headroom | Placement result |
|---|---|---|---|
| A | 300 kW | 180 kW | Not eligible Cooling binds |
| B | 260 kW | 260 kW | Review Only 20 kW nominal margin |
| C | 400 kW | 350 kW | Candidate Subject to distribution and redundancy checks |
Zone C is the strongest candidate on these two measurements. That does not establish that the job can run there. Network placement, physical rack density, the effect of cooling-unit maintenance and the operator's defined reserve policy may still rule it out. The scheduler should report these reasons to the engineer instead of silently substituting a neighboring rack.
Prevent oscillation and unsafe load ramps
Real-time algorithms can chase each other's actions. Moving a job to a cooler zone raises that zone's heat load; another scheduler cycle then sees the new temperature and moves the work back. This control oscillation can increase wear and reduce effective compute throughput even when no breaker trips.
Do not copy a universal limit such as “500 kW per minute.” Suitable ramps depend on equipment ratings, utility arrangements, electrical topology and a site's agreed operating procedures. Likewise, the building management system should not be forced to react to every scheduling change by altering primary control loops.
Who is allowed to act?
In owner-operated environments, IT scheduling teams can coordinate with facility operations using a documented interface. In colocation, customers may receive agreed read-only capacity signals, while the provider retains authority over shared electrical and cooling infrastructure. Granting an external job scheduler direct BMS write access is a separate, high-risk engineering decision—not a default feature of workload placement.
Record which team defines the capacity envelope, who may authorize exceptions, and what happens when telemetry fails. That governance can be captured in the AI system inventory along with the model owner, version, decision scope and data sources.
A small pilot that produces useful evidence
Start in advisory mode on a set of interruptible jobs. Compare recommended placements with operator decisions. Track job completion time, measured power at the relevant branch, inlet or coolant temperatures, rejected placements and rescheduling frequency. Only add automated placement after an engineer signs off on stale-data behavior and tested limits.
Success is not merely a lower average PUE. A useful outcome is a measurable reduction in scheduling delays or peak demand without exceeding the facility's tested electrical and thermal envelope.
Takeaway: the scheduler should use facility telemetry to make better placement proposals, while engineering protections remain independent of the model.
Sources and further reading
Primary frameworks and reference materials. Engineering decisions require site-specific validation; legal classifications require assessment of the actual use case.
- Powering AI and Data Center InfrastructureUS Department of Energy
- Guide to OT Security, SP 800-82 Rev. 3NIST
- AI RMF: Map, Measure and ManageNIST AI Resource Center