Carbon-aware scheduling for LLM batch jobs (grid intensity and flexibility)

Not every LLM workload needs an answer in milliseconds. Batch scoring, reprocessing, synthetic data generation, and offline evaluations can often shift in time or place. When electricity comes from a grid whose carbon intensity swings by hour and region, scheduling flexible compute against cleaner power is one of the few levers that reduces operational emissions without changing the model.

Average vs marginal intensity

Grid carbon intensity can be quoted as average (typical grid mix) or marginal (emissions from the next incremental kWh). Marginal signals are often used in carbon-aware scheduling research because they match short-term dispatch of plants. In practice, product teams may only have hourly regional averages from open data or cloud tools—document whichever you use and avoid mixing methods inside one KPI.

What the literature suggests

Studies on data centers and distributed clouds report meaningful CO₂ reductions when flexible jobs follow lower-carbon periods or regions—sometimes on the order of roughly ten percent for temporal shifting in surveyed AI workload settings, with higher potential when both time and place flex exist. Hyperscale operators have published carbon-intelligent scheduling that throttles or delays non-urgent work when local grids are dirty. The takeaway for LLM teams: batch APIs and queues are climate tools, not only cost tools.

How to implement without breaking SLAs

• Classify jobs into interactive vs deferrable; only the latter are candidates for shifting.
• Cap delay with explicit SLOs (e.g. “within 6 hours”) so product owners stay in control.
• Prefer regions with both low carbon intensity and acceptable data residency.
• Log energy or token usage alongside the schedule window so finance and sustainability can reconcile numbers—aligned with API-grade tracking.

Reporting boundaries

For CSRD-style disclosures, carbon-aware scheduling changes operational Scope 2 / market-based narratives when paired with PPAs or location choices—but only if your evidence chain matches the reporting rule you apply. When in doubt, pair operational metrics with GHG Protocol scope discipline.