Engines6 min read

vLLM vs SGLang on NVIDIA B300: eight models, head to head

We ran eight open models on both vLLM and SGLang on NVIDIA B300. SGLang was faster per user in 5 of 8; vLLM won peak throughput in 6 of 8.

Key takeaways

  • Across eight models run on both engines, vLLM won 14 of 24 comparisons and SGLang won 10.
  • SGLang had the higher single-user speed in 5 of 8 models; the gap was usually under 10%.
  • vLLM had the higher peak throughput in 6 of 8 models, by 47% on Qwen3.8-27B and 61% on GLM-5.3-Flash.
  • vLLM had the lower time to first token in 5 of 8 models.

Which inference engine is faster depends on what you mean by fast. We ran the same eight models on vLLM and on SGLang, on the same NVIDIA B300 hardware, and compared three things: how fast one user's response streams, how long that user waits for the first token, and how many tokens per second the server produces when it is fully loaded.

All figures are from one batch measured on NVIDIA B300 GPUs between 9 and 10 November 2026, with vLLM 0.32.0 and SGLang 0.5.22. Single-user speed and time to first token are medians at concurrency 1 on the chat workload (1,000 tokens in, 1,000 out).

Results by model

Model (GPUs)EngineTok/s per userTTFT (ms)Peak tok/s (users)
Qwen3.8-27B (1)vLLM91.6695,947 (256)
Qwen3.8-27B (1)SGLang95.6634,052 (128)
gemma-4-31B-it (1)vLLM77.2643,374 (256)
gemma-4-31B-it (1)SGLang82.4632,883 (128)
gemma-4-E4B-it (1)vLLM227.51918,468 (256)
gemma-4-E4B-it (1)SGLang252.54518,061 (256)
DeepSeek-V4-Pro (4)vLLM105.2875,811 (256)
DeepSeek-V4-Pro (4)SGLang103.92535,684 (256)
GLM-5.3 (4)vLLM111.2785,156 (256)
GLM-5.3 (4)SGLang118.0885,061 (256)
GLM-5.3-Flash (2)vLLM159.3636,375 (256)
GLM-5.3-Flash (2)SGLang172.0763,962 (256)
Mistral-Small-4-119B (1)vLLM280.7469,558 (256)
Mistral-Small-4-119B (1)SGLang269.6819,602 (256)
Mistral-Medium-3.5-128B (1)vLLM41.21881,618 (128)
Mistral-Medium-3.5-128B (1)SGLang40.91371,735 (256)
vLLM 0.32.0 and SGLang 0.5.22 on NVIDIA B300. Peak throughput is the best aggregate output rate across the concurrency sweep, with the concurrency where it occurred.

Single-user speed: SGLang, narrowly

SGLang streamed faster to a single user in five of the eight models. The margins are small, from 4% on Qwen3.8-27B to 11% on gemma-4-E4B-it. Where vLLM led (DeepSeek-V4-Pro, Mistral-Small-4 and Mistral-Medium-3.5) the margin was 1% to 4%. If your product is one user talking to one model, the engines are close enough that other factors should decide.

Peak throughput: vLLM, sometimes by a lot

Under load the picture changes. vLLM produced more total tokens per second in six of eight models. On Qwen3.8-27B it reached 5,947 tok/s at 256 concurrent users against SGLang's 4,052 at 128, a 47% lead. On GLM-5.3-Flash the lead was 61% (6,375 against 3,962). On the two Mistral models SGLang came out ahead, by 0.5% and 7%.

Peak throughput is what sets cost per token, because the GPU bill is the same whether it serves one user or 256. An engine that sustains 47% more tokens on the same GPU cuts the cost of each token by about a third.

Time to first token: vLLM in five of eight

vLLM responded sooner in five models. The largest gap was DeepSeek-V4-Pro on four GPUs: 87 ms on vLLM against 253 ms on SGLang. On gemma-4-E4B-it vLLM answered in 19 ms against 45 ms. SGLang was quicker on Qwen3.8-27B, gemma-4-31B-it (by 1 ms) and Mistral-Medium-3.5 (137 ms against 188 ms).

What this does and does not show

  • It is one hardware platform and one version of each engine. Both projects ship performance work frequently, so results will move.
  • Engines ran with their default settings for each model. Tuning either one can change the ranking.
  • Two more models in the batch ran on one engine only and are left out of this comparison. All runs, including failed ones, are listed on the leaderboard and in the methodology.

Frequently asked questions

Is vLLM or SGLang faster?
It depends on the metric. In our NVIDIA B300 batch, SGLang had higher single-user speed in 5 of 8 models, while vLLM had higher peak throughput in 6 of 8 and lower time to first token in 5 of 8.
Which engine is cheaper to run?
Cost per token follows peak throughput, since the GPU costs the same per hour at any load. vLLM reached higher peak throughput in 6 of 8 models, by up to 61%, so it was the cheaper engine per token for those models on this hardware.
What versions were tested?
vLLM 0.32.0 and SGLang 0.5.22, on NVIDIA B300 GPUs, measured on 9 and 10 November 2026.

Measured by ModelPerf.ai on our own hardware. How each number is produced is in the methodology; current results are on the leaderboard.

All analysis