Measured energy per request, on your workload.
Scope a measurementLoupe by LeanLM
Almost nobody measures it. Model cards report parameters, leaderboards report accuracy, and providers report price per token. None of those tell you the joules.
We measure it per request, on one server, at a stated boundary, across model sizes and serving configurations. The numbers below are what came back.
Your data doesn’t leave your environment. The method is published so you can check it.
24.3 J
Per request on a 2B model, against 284.5 J on a 27B
same task, same server, 3,072 cases
67%
Of that energy came from the output format, not the model
same model, constraint changed
51×
The energy reasoning cost for 2.6 points of accuracy
300-case banking test
The problem
Those two numbers move on their own. A request that looks cheap on an invoice can be expensive in energy, and the gap is invisible from either side of a metered API. If you self-host, the watts are your bill. If you’re power-capped or queued for a GPU allocation, the watts are your ceiling. Either way, nobody is handing you the figure.
The reason it goes unmeasured is that it’s fiddly rather than hard. It needs a fixed server, a warmed idle baseline, a stated boundary, and the discipline to hold everything else constant. That’s the whole job.
What we found
The obvious lever is model size, and it’s real: a 2B model used a twelfth of the energy of a 27B on the same intent task. The less obvious lever turned out to be larger in proportion. Holding the model fixed and changing only the output constraint moved per-request energy from 73.1 J to 24.3 J.
| What changed | Energy per request | Change |
|---|---|---|
| 27B model, label grammar | 284.5 J | baseline |
| 2B model, JSON schema | 73.1 J | −74% |
| 2B model, label grammar | 24.3 J | −91% |
A JSON schema makes the model generate braces, quoted keys and whitespace before it reaches the answer. Every one of those is a decode step, and decode steps are where the joules go. Swapping the schema for a restricted label grammar removed two thirds of the remaining energy without touching the weights.
Measured at batch size 1 on one reference server. We haven’t yet measured whether the format gap survives production batching, and a restricted grammar only applies where the output space is bounded.
How we measure
An energy number without these is not checkable, and most published ones omit at least half. Ours are stated on every result, including the ones we can’t yet fill in.
What read the power, and how often.
GPU only, or the whole node. These differ by a lot and are frequently conflated.
Whether a warmed idle baseline was subtracted, and what it was.
Serving engine, batch size, and the full server arguments.
Prompt tokens and generated tokens have very different energy profiles.
How a measured window was divided across requests.
Model-serving energy only. Retrieval, embedding and orchestration sit outside the boundary and are excluded rather than estimated. A model-serving figure is not a whole-system figure and we don’t present it as one.
What it’s for
The output-format result cost nothing to act on and removed two thirds of the energy on that task. On a saturated GPU that is serving capacity you get back without buying anything. Waste that size stays invisible until somebody puts a meter on it, and it tends not to be where people assume.
Questions
A JSON schema makes the model generate its own syntax: braces, quoted keys, whitespace. Each of those is a decode step, and decode is where most inference energy goes. A restricted label grammar emits the answer and stops. On the task we measured, the same model produced the same answers for a third of the energy.
If you self-host, yes, because you pay for the power and the GPU hours. On a metered API the relationship is indirect: you’re billed per token, and fewer output tokens is both fewer joules and a smaller invoice, but the provider’s margin sits between the two. We report the joules we measured and the token counts behind them, and leave the arithmetic on your contract to you.
Carefully, and often unfavourably. On Banking77 and ContractNLI there are task-specific encoder models under 350M parameters that score above the generative models we measured. That is itself an energy finding: running a general-purpose model where a specialised one would do has a measurable cost, and it is one of the things a measurement turns up.
It doesn’t have to. The measurement rig runs inside your VPC, or your engineers run it and we interpret the output. That keeps the third-party risk assessment and the data-processing agreement off the critical path, which is usually where this kind of work actually stalls.
A single reference server, stated on every result, with the GPU model and region named. We have replayed a run from AMD onto a rented NVIDIA GPU and matched 298 of 300 items; the two that didn’t match are reported rather than rounded away. Portability across hardware needs its own evidence and we don’t assert it.
Fixed scope and fixed fee for one workload, quoted after a short scoping call, with no ongoing dependency.
Scope a measurement
Tell us what you run and where it runs. If measuring it won’t tell you anything useful, we’ll say so on the call.
Five fields, no attachments, and we’re not asking for traces at this stage. Security posture, sub-processors and GPU region come up on the call before anything technical starts.