vLLM 0.32.0
NVIDIA B300 x1 · tp1 × pp1Eco-friendlyFastest
- Speed per user (c=1)
- 280.7tok/s
- Time to first token
- 46ms
- Peak throughput
- 9,558tok/s @ 256
- Per GPU at peak
- 9,558tok/s
- Cold start
- 406s
- Context tested
- 8,192tokens
Measured 10 Nov 2026 · Throughput sweep