Leaderboard
Measured inference performance
Measured on our own hardware: single-user speed and time to first token at concurrency 1, and peak throughput at the best clean concurrency level (chat, 1k tokens in / 1k out). Performance only, no accuracy scores.
Updated 21 Nov 2026, 14:24 UTC
Greenest
gemma-4-E4B-it
vLLM · NVIDIA B300 x1
Included in Pro
Lowest GPU energy per million output tokens
Most profitable
gemma-4-E4B-it
vLLM · NVIDIA B300 x1
Included in Pro
Most output tokens per dollar of GPU time
Fastest response
Mistral-Small-4-119B-2603
vLLM · NVIDIA B300 x1
280.7 tok/s per user
Highest single-user generation speed
Best throughput per GPU
gemma-4-E4B-it
vLLM · NVIDIA B300 x1
18,467.7 tok/s per GPU
Most output tokens per second per GPU
| # | Model | Hardware | Tok/s per user | TTFT | Peak tok/s | $ / 1M tok | Tok/s per W | Wh / 1M tok |
|---|---|---|---|---|---|---|---|---|
| 1 | Mistral-Small-4-119B-2603 mistralai · vLLM 0.32.0 Eco-friendlyFastest | NVIDIA B300 x1 tp1 × pp1 | 280.7 | 46 ms | 9,558 @ 256 users | Pro | Pro | Pro |
| 2 | Mistral-Small-4-119B-2603 mistralai · SGLang 0.5.22 Eco-friendlyProfitable | NVIDIA B300 x1 tp1 × pp1 | 269.6 | 81 ms | 9,602 @ 256 users | Pro | Pro | Pro |
| 3 | gemma-4-E4B-it google · SGLang 0.5.22 Eco-friendlyProfitable | NVIDIA B300 x1 tp1 × pp1 | 252.5 | 45 ms | 18,061 @ 256 users | Pro | Pro | Pro |
| 4 | gemma-4-E4B-it google · vLLM 0.32.0 Eco-friendlyProfitableTop throughput | NVIDIA B300 x1 tp1 × pp1 | 227.5 | 19 ms | 18,468 @ 256 users | Pro | Pro | Pro |
| 5 | DeepSeek-V4.1-Flash deepseek-ai · vLLM 0.32.0 | NVIDIA B300 x4 tp4 × pp1 | 192.7 | 33 ms | 12,730 @ 256 users | Pro | Pro | Pro |
| 6 | GLM-5.3-Flash zai-org · SGLang 0.5.22 | NVIDIA B300 x2 tp2 × pp1 | 172.0 | 76 ms | 3,962 @ 256 users | Pro | Pro | Pro |
| 7 | GLM-5.3-Flash zai-org · vLLM 0.32.0 | NVIDIA B300 x2 tp2 × pp1 | 159.3 | 63 ms | 6,375 @ 256 users | Pro | Pro | Pro |
| 8 | GLM-5.3 zai-org · SGLang 0.5.22 | NVIDIA B300 x4 tp4 × pp1 | 118.0 | 88 ms | 5,061 @ 256 users | Pro | Pro | Pro |
| 9 | GLM-5.3 zai-org · vLLM 0.32.0 | NVIDIA B300 x4 tp4 × pp1 | 111.2 | 78 ms | 5,156 @ 256 users | Pro | Pro | Pro |
| 10 | DeepSeek-V4-Pro deepseek-ai · vLLM 0.32.0 | NVIDIA B300 x4 tp4 × pp1 | 105.2 | 87 ms | 5,811 @ 256 users | Pro | Pro | Pro |
| 11 | DeepSeek-V4-Pro deepseek-ai · SGLang 0.5.22 | NVIDIA B300 x4 tp4 × pp1 | 103.9 | 253 ms | 5,684 @ 256 users | Pro | Pro | Pro |
| 12 | Qwen3.8-27B Qwen · SGLang 0.5.22 | NVIDIA B300 x1 tp1 × pp1 | 95.6 | 63 ms | 4,052 @ 128 users | Pro | Pro | Pro |
| 13 | Qwen3.8-27B Qwen · vLLM 0.32.0 Eco-friendly | NVIDIA B300 x1 tp1 × pp1 | 91.6 | 69 ms | 5,947 @ 256 users | Pro | Pro | Pro |
| 14 | gemma-4-31B-it google · SGLang 0.5.22 | NVIDIA B300 x1 tp1 × pp1 | 82.4 | 63 ms | 2,883 @ 128 users | Pro | Pro | Pro |
| 15 | gemma-4-31B-it google · vLLM 0.32.0 | NVIDIA B300 x1 tp1 × pp1 | 77.2 | 64 ms | 3,374 @ 256 users | Pro | Pro | Pro |
| 16 | Mistral-Medium-3.5-128B mistralai · vLLM 0.32.0 | NVIDIA B300 x1 tp1 × pp1 | 41.2 | 188 ms | 1,618 @ 128 users | Pro | Pro | Pro |
| 17 | Mistral-Medium-3.5-128B mistralai · SGLang 0.5.22 | NVIDIA B300 x1 tp1 × pp1 | 40.9 | 137 ms | 1,735 @ 256 users | Pro | Pro | Pro |
| 18 | Kimi-K3 moonshotai · vLLM 0.32.0 Partial run | NVIDIA B300 x7 tp1 × pp7 | 38.1 | 322 ms | 441 @ 32 users | Pro | Pro | Pro |
ModelPerf Pro
Unlock cost, energy and concurrency curves for every model here
- Cost per million tokens for every model, engine and GPU
- Energy per million tokens, tokens per watt and GPU power draw
- Full concurrency curves for all three workload scenarios
- Unlimited ROI and break-even calculations with PDF export
- CSV export of the full leaderboard, paid metrics included
- New models and re-runs included as they are published
$39 / month, billed annually
Get ModelPerf Proor $49 month to month
Or unlock a single model for $10 from its page.
Compare plans