Leaderboard

Measured inference performance

Measured on our own hardware: single-user speed and time to first token at concurrency 1, and peak throughput at the best clean concurrency level (chat, 1k tokens in / 1k out). Performance only, no accuracy scores.

Updated 21 Nov 2026, 14:24 UTC

Greenest
gemma-4-E4B-it
vLLM · NVIDIA B300 x1
Included in Pro

Lowest GPU energy per million output tokens

Most profitable
gemma-4-E4B-it
vLLM · NVIDIA B300 x1
Included in Pro

Most output tokens per dollar of GPU time

Fastest response
Mistral-Small-4-119B-2603
vLLM · NVIDIA B300 x1
280.7 tok/s per user

Highest single-user generation speed

Best throughput per GPU
gemma-4-E4B-it
vLLM · NVIDIA B300 x1
18,467.7 tok/s per GPU

Most output tokens per second per GPU

Export CSV · Pro
#ModelHardwareTok/s per userTTFTPeak tok/s$ / 1M tokTok/s per WWh / 1M tok
1Mistral-Small-4-119B-2603
mistralai · vLLM 0.32.0
Eco-friendlyFastest
NVIDIA B300 x1
tp1 × pp1
280.746 ms9,558
@ 256 users
ProProPro
2Mistral-Small-4-119B-2603
mistralai · SGLang 0.5.22
Eco-friendlyProfitable
NVIDIA B300 x1
tp1 × pp1
269.681 ms9,602
@ 256 users
ProProPro
3gemma-4-E4B-it
google · SGLang 0.5.22
Eco-friendlyProfitable
NVIDIA B300 x1
tp1 × pp1
252.545 ms18,061
@ 256 users
ProProPro
4gemma-4-E4B-it
google · vLLM 0.32.0
Eco-friendlyProfitableTop throughput
NVIDIA B300 x1
tp1 × pp1
227.519 ms18,468
@ 256 users
ProProPro
5DeepSeek-V4.1-Flash
deepseek-ai · vLLM 0.32.0
NVIDIA B300 x4
tp4 × pp1
192.733 ms12,730
@ 256 users
ProProPro
6GLM-5.3-Flash
zai-org · SGLang 0.5.22
NVIDIA B300 x2
tp2 × pp1
172.076 ms3,962
@ 256 users
ProProPro
7GLM-5.3-Flash
zai-org · vLLM 0.32.0
NVIDIA B300 x2
tp2 × pp1
159.363 ms6,375
@ 256 users
ProProPro
8GLM-5.3
zai-org · SGLang 0.5.22
NVIDIA B300 x4
tp4 × pp1
118.088 ms5,061
@ 256 users
ProProPro
9GLM-5.3
zai-org · vLLM 0.32.0
NVIDIA B300 x4
tp4 × pp1
111.278 ms5,156
@ 256 users
ProProPro
10DeepSeek-V4-Pro
deepseek-ai · vLLM 0.32.0
NVIDIA B300 x4
tp4 × pp1
105.287 ms5,811
@ 256 users
ProProPro
11DeepSeek-V4-Pro
deepseek-ai · SGLang 0.5.22
NVIDIA B300 x4
tp4 × pp1
103.9253 ms5,684
@ 256 users
ProProPro
12Qwen3.8-27B
Qwen · SGLang 0.5.22
NVIDIA B300 x1
tp1 × pp1
95.663 ms4,052
@ 128 users
ProProPro
13Qwen3.8-27B
Qwen · vLLM 0.32.0
Eco-friendly
NVIDIA B300 x1
tp1 × pp1
91.669 ms5,947
@ 256 users
ProProPro
14gemma-4-31B-it
google · SGLang 0.5.22
NVIDIA B300 x1
tp1 × pp1
82.463 ms2,883
@ 128 users
ProProPro
15gemma-4-31B-it
google · vLLM 0.32.0
NVIDIA B300 x1
tp1 × pp1
77.264 ms3,374
@ 256 users
ProProPro
16Mistral-Medium-3.5-128B
mistralai · vLLM 0.32.0
NVIDIA B300 x1
tp1 × pp1
41.2188 ms1,618
@ 128 users
ProProPro
17Mistral-Medium-3.5-128B
mistralai · SGLang 0.5.22
NVIDIA B300 x1
tp1 × pp1
40.9137 ms1,735
@ 256 users
ProProPro
18Kimi-K3
moonshotai · vLLM 0.32.0
Partial run
NVIDIA B300 x7
tp1 × pp7
38.1322 ms441
@ 32 users
ProProPro

ModelPerf Pro

Unlock cost, energy and concurrency curves for every model here

  • Cost per million tokens for every model, engine and GPU
  • Energy per million tokens, tokens per watt and GPU power draw
  • Full concurrency curves for all three workload scenarios
  • Unlimited ROI and break-even calculations with PDF export
  • CSV export of the full leaderboard, paid metrics included
  • New models and re-runs included as they are published
$39 / month, billed annually

or $49 month to month

Get ModelPerf Pro

Or unlock a single model for $10 from its page.

Compare plans