Independent benchmarking
Benchmark your model on the hardware you care about
Commission independent performance, cost, and energy analysis. Reproducible methodology, raw data included.
Plans
Choose your plan
Single Run
One model on one hardware configuration
Contact for pricing
- Full benchmark suite (speed, latency, throughput)
- Cost per token analysis
- Energy efficiency measurement
- Turnaround time discussed during consultation
- PDF report + raw logs
- Publishable results (optional)
Hardware Matrix
One model across multiple hardware configs
Contact for pricing
- Multiple hardware configurations
- Comparative performance analysis
- TCO modeling and recommendations
- Priority execution
- Detailed written analysis
- Private or public results
Enterprise
Ongoing benchmarking partnership
Contact for pricing
- Custom model × hardware testing
- Private deployment benchmarking
- Dedicated support
- Regular reports and consulting
- API access to results
- Custom arrangements available
Process
How it works
- Step 1
Submit request
Tell us your model (weights, endpoint, or Hugging Face repo) and target hardware.
- Step 2
We run tests
The full benchmark suite: speed, latency, throughput, energy and cost.
- Step 3
Receive analysis
A PDF report with charts, methodology, raw logs, and recommendations.
- Step 4
Publish (optional)
Add your results to our public leaderboard for community visibility.
Scope
What we measure
Performance
- Tokens per second (output speed)
- Time to first token (latency)
- Peak throughput at saturation
- Concurrency behaviour (chat swept from 1 to 256 users)
- Cold start time
Economics
- Cost per million output tokens
- GPU power draw (idle and peak)
- Energy per million tokens (Wh)
- Tokens per second per watt
- TCO projections and break-even analysis
All tests run on vLLM and SGLang with standard configs. Workloads cover chat, prefill-heavy, and decode-heavy scenarios. Energy is GPU power read from the GPU's own telemetry. Results cover performance, cost and energy; they do not include accuracy scores. Raw logs and configurations are included with your report.
Get started
Request a benchmark
Fill out the form and we'll get back to you within 72 hours with a detailed quote and timeline.
- Results stay private unless you choose to publish
- Raw logs and configurations included
- Paid benchmarks are always disclosed
FAQ
Frequently asked questions
How long does a benchmark take?
Turnaround time depends on hardware availability and workload complexity. We'll provide an estimated timeline during the initial consultation. Contact us to discuss your specific requirements.
Can I benchmark a private model?
Yes. We support private models via secure weight transfer, API endpoints, or on-premises deployment if you have specific security requirements. Private benchmarks include an NDA.
What hardware do you support?
Our benchmark runner supports NVIDIA (CUDA) and AMD (ROCm) GPU hosts with the vLLM and SGLang engines, and it can also benchmark hosted endpoints over HTTPS. Published results today are on NVIDIA B300. If you need other hardware, tell us in your request and we will confirm what we can arrange.
Will my results be published publicly?
Only if you choose. All commissioned benchmarks default to private. You can opt to publish results on our leaderboard for community visibility.
How do you ensure reproducibility?
Your report states the exact vLLM or SGLang version, the hardware and the workload for every number, and includes the raw logs and configurations from your runs, so the measurements can be repeated on the same hardware.
Do you accept payment from hardware vendors?
No. We're funded by Podstack and benchmark commissions. We don't accept payment from hardware vendors, cloud providers, or model developers in exchange for favorable results. Paid benchmarks are always disclosed.