Independent benchmarking

Benchmark your model on the hardware you care about

Commission independent performance, cost, and energy analysis. Reproducible methodology, raw data included.

Read the methodology

Plans

Choose your plan

Single Run

One model on one hardware configuration

Contact for pricing

  • Full benchmark suite (speed, latency, throughput)
  • Cost per token analysis
  • Energy efficiency measurement
  • Turnaround time discussed during consultation
  • PDF report + raw logs
  • Publishable results (optional)

Hardware Matrix

One model across multiple hardware configs

Contact for pricing

  • Multiple hardware configurations
  • Comparative performance analysis
  • TCO modeling and recommendations
  • Priority execution
  • Detailed written analysis
  • Private or public results

Enterprise

Ongoing benchmarking partnership

Contact for pricing

  • Custom model × hardware testing
  • Private deployment benchmarking
  • Dedicated support
  • Regular reports and consulting
  • API access to results
  • Custom arrangements available

Process

How it works

  1. Step 1

    Submit request

    Tell us your model (weights, endpoint, or Hugging Face repo) and target hardware.

  2. Step 2

    We run tests

    The full benchmark suite: speed, latency, throughput, energy and cost.

  3. Step 3

    Receive analysis

    A PDF report with charts, methodology, raw logs, and recommendations.

  4. Step 4

    Publish (optional)

    Add your results to our public leaderboard for community visibility.

Scope

What we measure

Performance

  • Tokens per second (output speed)
  • Time to first token (latency)
  • Peak throughput at saturation
  • Concurrency behaviour (chat swept from 1 to 256 users)
  • Cold start time

Economics

  • Cost per million output tokens
  • GPU power draw (idle and peak)
  • Energy per million tokens (Wh)
  • Tokens per second per watt
  • TCO projections and break-even analysis

All tests run on vLLM and SGLang with standard configs. Workloads cover chat, prefill-heavy, and decode-heavy scenarios. Energy is GPU power read from the GPU's own telemetry. Results cover performance, cost and energy; they do not include accuracy scores. Raw logs and configurations are included with your report.

Get started

Request a benchmark

Fill out the form and we'll get back to you within 72 hours with a detailed quote and timeline.

  • Results stay private unless you choose to publish
  • Raw logs and configurations included
  • Paid benchmarks are always disclosed

e.g., meta-llama/Llama-3.3-70B-Instruct or an API endpoint

By submitting, you agree we may contact you about your benchmark request. We will not share your information with third parties.

FAQ

Frequently asked questions

How long does a benchmark take?

Turnaround time depends on hardware availability and workload complexity. We'll provide an estimated timeline during the initial consultation. Contact us to discuss your specific requirements.

Can I benchmark a private model?

Yes. We support private models via secure weight transfer, API endpoints, or on-premises deployment if you have specific security requirements. Private benchmarks include an NDA.

What hardware do you support?

Our benchmark runner supports NVIDIA (CUDA) and AMD (ROCm) GPU hosts with the vLLM and SGLang engines, and it can also benchmark hosted endpoints over HTTPS. Published results today are on NVIDIA B300. If you need other hardware, tell us in your request and we will confirm what we can arrange.

Will my results be published publicly?

Only if you choose. All commissioned benchmarks default to private. You can opt to publish results on our leaderboard for community visibility.

How do you ensure reproducibility?

Your report states the exact vLLM or SGLang version, the hardware and the workload for every number, and includes the raw logs and configurations from your runs, so the measurements can be repeated on the same hardware.

Do you accept payment from hardware vendors?

No. We're funded by Podstack and benchmark commissions. We don't accept payment from hardware vendors, cloud providers, or model developers in exchange for favorable results. Paid benchmarks are always disclosed.