Training vs Inference: Where AI Energy Really Goes
On this page
Ask where AI's energy goes and you get two very different answers depending on whether you mean building the model or using it. The training vs inference energy question splits neatly along the AI lifecycle: one phase is a concentrated upfront burst, the other is a steady trickle that never stops. Both matter, but for different reasons.
This guide separates the two, shows how each scales, and explains why per-request measurement is the number users should actually watch. The short version: training makes the headlines, but inference is what grows with every new person who signs up.
Splitting the AI lifecycle
An AI model's energy story has two chapters. Training is the learning phase, where chips run flat out for weeks to fit the model's parameters. Inference is the serving phase, where the finished model responds to prompts. They draw power in completely different shapes: training is a tall, brief spike, while inference is a long, flat line that extends for as long as the model is in service.
Both run inside data centers, so both carry the same overhead for cooling and power delivery, captured by a PUE of roughly 1.56. And both convert to carbon through grid intensity, around 0.395 kg CO2 per kWh on the US grid. The difference is entirely in duration and repetition.
The training side: one big spike
Training is expensive because it repeats a vast calculation millions of times until the model converges. For a large model this can mean thousands of processors running continuously, adding up to a substantial one-off carbon cost. We cover that number in detail in the carbon footprint of training AI models.
The important thing about training energy is that it is fixed. It does not grow when the model becomes popular. Whether one person or a hundred million people use the result, the training bill was already paid. That makes it a poor guide to the impact of your own daily usage.
The inference side: energy that scales with adoption
Inference behaves the opposite way. Each request is cheap, a typical text answer sitting somewhere under a gram to a few grams of CO2, but the total climbs directly with the number of users and how often they prompt. Double the audience and you roughly double the inference energy.
Per-token costs vary a lot by model class. A small model might use around 0.0008 Wh per token, a mid-sized one about 0.0015 Wh, and a frontier or reasoning model roughly 0.0038 to 0.0040 Wh. That spread is why model choice can change the energy of a single answer several times over. See the breakdown in AI energy usage by model and the extra cost of step-by-step models in reasoning models energy cost.
- Small model: about 0.0008 Wh per token
- Mid model: about 0.0015 Wh per token
- Frontier model: about 0.0038 Wh per token
- Reasoning model: about 0.0040 Wh per token
When inference overtakes training
Because training is fixed and inference keeps accumulating, there is a crossover point where total serving energy passes the original training cost. For a lightly used model that point may never arrive. For a mainstream assistant answering hundreds of millions of prompts a day, it arrives fast, and from then on inference is the dominant slice of the model's lifetime energy.
This reframes the whole debate. The training number is real but static. The inference number is the one still growing, and it grows because of collective demand, including yours.
Why per-request measurement matters most
If inference is the part that scales with you, then the useful metric is the energy of your individual requests, not the training statistics you cannot change. Measuring per request turns a vague concern into a concrete, improvable number. You can then choose smaller models, trim unnecessary regenerations, and see the effect immediately. Our guide on how to reduce your AI carbon footprint walks through the tactics.
Ecoia is built around this idea. Every request is measured for energy, carbon and water, shown transparently, and then more than offset: the platform retires verified offsets beyond 200 percent of the measured impact. You can see the mechanism on the how it works page or estimate your own usage with the carbon footprint calculator.
The headline: Training is a fixed one-time spike while inference scales with every user, so the number that matters for you is the measured energy of each request.
FAQ
What is the difference between training and inference energy?
Training energy is the one-time cost of building a model by fitting its parameters, concentrated into a few weeks of heavy compute. Inference energy is the recurring cost of running the finished model to answer prompts. Training is fixed; inference grows with usage.
Does inference really use more energy than training?
For a widely used model, yes, over its lifetime. Any single inference is tiny, but repeated across billions of requests the total can exceed the original training cost. For rarely used models the training cost may never be surpassed.
Which one can I actually influence?
Inference. Training is finished before you use a model, but you control every request you make. Choosing a right-sized model and prompting efficiently directly lowers your inference energy.
Why measure energy per request?
Because per-request figures reflect the part of AI energy that scales with your behaviour. They turn an abstract worry into a number you can watch and reduce, and they let a platform offset your exact measured impact.