vLLM 0.32.0
NVIDIA B300 x4 · tp4 × pp1- Speed per user (c=1)
- 105.2tok/s
- Time to first token
- 87ms
- Peak throughput
- 5,811tok/s @ 256
- Per GPU at peak
- 1,453tok/s
- Cold start
- 1,349s
- Context tested
- 8,192tokens
Measured 9 Nov 2026 · Throughput sweep
Measured 9 Nov 2026 · Throughput sweep
Measured 9 Nov 2026 · Throughput sweep
ModelPerf Pro