AI's Energy Footprint
AI is an energy-intensive technology at a scale that is now attracting board-level attention, government policy, and investment in new power generation. Understanding the numbers - and why they matter - is increasingly important for builders making architectural decisions that have environmental and cost implications.
Numbers That Matter
- A single ChatGPT query uses approximately 10× the energy of a Google Search query
- Training GPT-4 is estimated at 50 GWh of electricity - enough to power ~4,500 US homes for a year
- A single NVIDIA H100 GPU consumes 700W under full load. A server with 8 H100s (DGX H100) draws 10.2 kW
- A data center with 10,000 H100s - a medium-sized AI training cluster - needs roughly 72 MW of continuous power
- Global AI electricity consumption is projected to reach 250–500 TWh per year by 2027 - comparable to a mid-sized European country
- By 2026, AI data centers in the US alone are projected to consume more electricity than some entire countries consume today
Training vs Inference Energy
Training (one-time): GPT-4: ~50 GWh over several months Llama 3 70B: ~3-5 GWh estimated Cost: enormous, but amortized across all future uses Runs: once (or a few times) per model Inference (continuous): Per query: tiny (fractions of a Wh) But multiplied by: billions of queries per day Total: exceeds training energy within weeks of deployment At ChatGPT's reported scale (~100M daily users): ~100M queries/day × 0.001 kWh/query = 100,000 kWh/day = 36.5 GWh/year - already comparable to training cost And growing.
The key insight: inference now dominates total AI energy consumption. Training is a one-time cost. Inference is continuous and scales with adoption. This is why inference efficiency (quantization, model routing, caching, smaller models) has become both a business and environmental imperative.
Per-Query Energy by Model Type
| Model / Task | Est. Energy per Query | Relative to Google Search |
|---|---|---|
| Google Search | ~0.0003 kWh | 1× (baseline) |
| ChatGPT (GPT-4o, short query) | ~0.001–0.003 kWh | 3–10× |
| Claude Opus (long reasoning) | ~0.005–0.015 kWh | 15–50× |
| Image generation (SDXL) | ~0.002–0.01 kWh | 7–33× |
| Video generation (10s clip) | ~0.05–0.5 kWh | 170–1700× |
| On-device SLM (Phi-3 on phone) | ~0.00003 kWh | 0.1× |
Estimates based on published data and GPU power measurements; actual values vary by implementation and infrastructure.
Why This Matters for Builders
Energy consumption translates directly to cost. A query to GPT-4 costs more not just because of the model license, but because of the underlying electricity and cooling infrastructure. Builders have real levers:
- Model routing - send simple queries to small models (10–100× less energy per query)
- Caching - a cached response uses essentially zero inference energy
- On-device inference - shifts energy cost to the user's existing device; marginal energy is often zero
- Batching - GPU utilization from 30% to 90% means ~3× more queries per kWh
- Quantization - INT4 vs FP16 roughly halves memory bandwidth, which is often the energy bottleneck
These are not just environmental choices - they are cost optimization choices that can reduce your AI infrastructure bill by 50–90%.
Energy Efficiency Trajectory
The good news: the energy cost per token has been falling rapidly. The cost to generate 1M tokens with a frontier-quality model has dropped roughly 100× in three years (2022–2025), driven by model efficiency improvements, hardware improvements, and competition. If this trend continues, the absolute energy consumption of AI may stabilize even as usage grows - but current projections suggest adoption growth is outpacing efficiency gains through at least 2027.