Energy per request,
measured.
Two public workloads, three serving configurations, one reference server. Every figure in the main table is model-serving energy at batch size 1; a batched run follows below it. We ran all of it ourselves, and nobody outside this project has reproduced it yet.
Version 1.1 · 8 October 2026 · next refresh by 8 November 2026
The measurements
Joules per request.
| Workload | Model | Output constraint | Accuracy | Energy / request |
|---|---|---|---|---|
| Banking77 | 27B | none | 93.9% | 284.5 J |
| Banking77 | 2B | JSON schema | 91.5% | 73.1 J |
| Banking77 | 2B | label grammar | 92.0% | 24.3 J |
| ContractNLI | 27B | — | 78.5% | 649.1 J |
| ContractNLI | 2B | — | 84.7% | 9.0 J |
Swipe the table for more columns.
In one sentence: on a 77-class intent task, the same 2B model answered the same 3,072 cases for 24.3 J per request under a restricted label grammar and 73.1 J under a JSON schema. That's a 67% energy reduction with no change to the model, the prompt or the task. The 27B incumbent on that task, with no output constraint, used 284.5 J. On ContractNLI, 1,763 cases, the 2B used 9.0 J against the 27B’s 649.1 J.
Banking77 is 3,072 cases, ContractNLI is 1,763. The spread on Banking77 is the useful part: the same 2B model, on the same task, moved from 73.1 J to 24.3 J when the output constraint changed from a JSON schema to a restricted label grammar. Two thirds of the JSON-schema run's energy went to generating punctuation. Shared prompts waste energy the same way: reusing a cached prefix cut energy per request 68 to 82% on 32,000-token prompts (prefix caching speed and energy).
Reading the accuracy column
Neither model is competitive on accuracy.
Both benchmarks have published task-specific baselines that score higher than either model above, at a fraction of the size. Banking77 was introduced in a 2020 paper whose whole argument was that small sentence encoders beat fine-tuning BERT-Large on intent detection. ContractNLI’s origin paper reports Span NLI BERT-base at 110M parameters reaching 83.8%, and a 335M variant reaching 87.5%.
Both generative models were the wrong tool for these two tasks. A small fine-tuned encoder beats them on accuracy, and the energy column shows what running a general-purpose model in its place costs. You only see that cost if you meter it.
Published baselines above are task-fine-tuned encoders with span-level heads. Our runs use a different configuration, so the comparison isn’t like-for-like. We cite them because they bound what’s achievable on these tasks.
Under batching
Does the format gap survive batching?
The format gap holds under batching, narrower: on vLLM with Qwen2.5-1.5B, constrained choice used 48% less energy per request than a JSON schema one request at a time, and 36% less at 32 concurrent. Batching itself cut energy per request 11 to 14 times.
| Concurrent requests | JSON schema | Constrained choice | Choice uses less |
|---|---|---|---|
| 1 | 8.10 J | 4.25 J | 48% (reps 47–48%) |
| 8 | 1.81 J | 0.86 J | 52% (reps 42–60%) |
| 32 | 0.59 J | 0.38 J | 36% (reps 34–38%) |
Swipe the table for more columns.
Each cell is the mean of two reps, in active joules per request. We ran vLLM 0.31.0 with Qwen2.5-1.5B-Instruct on one rented NVIDIA GPU, at temperature 0. Each window was the same 3,072 Banking77 prompts as the published run, repeated 5 times (15,360 requests). Power is GPU board power from nvidia-smi at 5 Hz, with the warm idle draw (87.2 W, median) subtracted. Joules per request is the window’s active energy divided by completed requests.
Accuracy was 92.0% for choice and 87.6% for the JSON schema. Choice produced 5 output tokens per request against 12. Labels agreed 99.6 to 99.9% between concurrency 1 and 32 (rep 1). Both concurrency-1 windows in rep 1 had one power-sampling gap of about 6 seconds inside 520 to 880 second windows; rep 2 had none, and those cells match rep 2 within 2%. At 32 concurrent, throughput was 298 requests per second for JSON and 395 for choice.
Concurrency 8 is noisy: the two reps landed anywhere from 42% to 60%, so read that row loosely. Batching from 1 to 32 cut energy per request 14 times for JSON and 11 times for choice. These joules come from a different model, engine and GPU than the main table, so only the ratios compare.
Disclosure
What every figure above discloses.
Boundary
Model serving only. Retrieval, embedding and orchestration are outside the boundary, and we don’t estimate them.
Concurrency
Batch size 1 for the main table. We re-measured the format gap under batching on 2026-10-08 (see below).
Hardware
One reference server. A replay from AMD onto a rented NVIDIA GPU matched 298 of 300 items, and the two mismatches are reported. Memory needs by model size were measured separately, on a rented 96 GB card: see VRAM by model size, measured.
Model family
Two families. The 2B is Gemma 4 E2B and the 27B is Qwen 3.8 27B. Nothing here establishes that the pattern holds across other models, and we don’t claim it does.
Measurement tool and sampling rate
The GPU’s on-board power sensor (amdgpu hwmon power1_average), sampled at 20 Hz (19.8 Hz achieved). Coding results elsewhere on this site sample at 5 Hz.
Idle handling and attribution
Warm idle, with the model loaded and no requests in flight (12.56 W for the 2B, 12.01 W for the 27B), is subtracted before integrating. One power window spans each run, and joules per request is that total divided by the request count, so time between requests is included. Serving engine: llama-server, one request at a time.
Controls
What the screening caught.
One result withdrawn before publication
A third workload produced a result that our pre-registered screening rejected because of a data defect. We withdrew it. That’s why this page has two workloads.
Reasoning: 51× for 2.6 points
Across 2,400 cases and eight public tasks, reasoning solved 5% more cases than not reasoning. On a 300-case banking test it cost 51 times the energy for a 2.6-point accuracy gain. Worth having for hard cases, expensive as a default.
Code: the retry loop, then the weights
On HumanEval+, a stock 7B coding model with a five-tier retry ladder passed 144 of 164 hidden tests; a 132-sample fine-tune matched it within noise at 17.3% fewer tokens and 20.4% less time per task. A follow-up run metered it: 1,367 J per task with the ladder against 728 J for a single attempt. The write-up, with the legend. A follow-up put small vs large models on the same 164 tasks: the 7B passed 144, a 27B passed 152.
Reproduce the format result yourself.
The JSON-schema versus label-grammar comparison is the one to run on your own stack. It needs no vendor and no data sharing, and an engineer can do it in an afternoon. If it holds for you, check which of your other workloads it applies to.
Follow the measurements
This page changes roughly monthly.
Each refresh adds workloads, fills in a disclosure field we couldn’t fill before, or corrects something. One email when that happens, nothing else.
No spam. Unsubscribe in one click.