vLLM 0.32.0
NVIDIA B300 x7 · tp1 × pp7Partial run
- Speed per user (c=1)
- 38.1tok/s
- Time to first token
- 322ms
- Peak throughput
- 441tok/s @ 32
- Per GPU at peak
- 63tok/s
- Cold start
- 1,414s
- Context tested
- 8,192tokens
Engine stopped at concurrency 64; only levels up to 32 were measured.
Measured 9 Nov 2026 · Throughput sweep