OneInfer benchmarks
Every number, with its source and date.
This page backs up OneInfer’s numbers. Prices come live from our public pricing API, stamped with the time they were fetched. Model scores and speeds come from Artificial Analysis, an independent benchmarking service, as stored in our model catalog. Every chart names its source.
Use it to compare what models cost with how they score, see how far one model’s price varies between providers, and work out what routing simple requests to a cheaper model saves on your traffic. Our own latency and kernel results are not published yet; the method and script are below, so you can run the same test.
How a latency result gets published, in 4 steps
Model ids from the catalog and the prompt sizes to test: about 1,000 or 10,000 input tokens, up to 400 output tokens.
--models anthropic/claude-haiku-5.5,openai/gpt-oss-120bNo run is published yet. Every result will follow these steps.
Scores and speed from Artificial Analysis
Artificial Analysis is an independent service that runs the same evaluations on every model and measures the speed of model APIs. These are its values for models OneInfer lists, next to our live prices.
Intelligence Index against price
Each dot is a model OneInfer lists: up is a higher Artificial Analysis Intelligence Index, right is a higher price on OneInfer (log scale). 37 models have this score.
- Highest score at its price or lower
- Other models
Intelligence Index
Price on OneInfer per 1M tokens, blended 3 parts input to 1 part output (log scale)
Source: Artificial Analysis, via OneInfer’s catalog. Fetched 21 October 2026, 22:02 UTC.
How it is measured: Artificial Analysis Intelligence Index methodology
Show the data as a tableHide the table
| Model | Intelligence Index | Coding Index | Output speed, tokens/s | Lowest price, in / out | Blended 3:1 |
|---|---|---|---|---|---|
| Claude Fable 5.1Anthropic | 54.5 | 79.1 | 53.5 | $10.00 / $50.00 | $20.00 |
| Muse Spark 1.3Meta | 53 | 76.3 | 221 | $1.25 / $4.25 | $2.00 |
| Kimi K3Novita | 50.2 | 76.2 | 42.2 | $3.00 / $15.00 | $6.00 |
| Grok 4.6xAI | 49.9 | 75.9 | 60.5 | $2.00 / $6.00 | $3.00 |
| GPT 6 AstraOpenAI | 47.8 | 76.2 | No value | $10.00 / $50.00 | $20.00 |
| Gemini 3.8 FlashGoogle | 47.1 | 76.3 | 355.8 | $0.75 / $3.75 | $1.50 |
| Qwen3.8 MaxNovita | 46.9 | 71.8 | 38.1 | $2.00 / $6.00 | $3.00 |
| Muse Spark 1.2Meta | 46.8 | 72.2 | 262 | $1.25 / $4.25 | $2.00 |
| Qwen3.8 FlashNovita | 45.6 | 73.1 | 76.4 | $0.15 / $0.47 | $0.23 |
| Grok 4.5xAI | 45.5 | 72.4 | 52.6 | $2.00 / $6.00 | $3.00 |
| Gemini 3.7 FlashGoogle | 45.2 | 76.1 | 324.6 | $0.75 / $3.75 | $1.50 |
| DeepSeek V4 ProNovita | 42.1 | 68.8 | 62.1 | $1.60 / $3.20 | $2.00 |
| Muse Spark 1.1Meta | 41.2 | 71.3 | No value | $1.25 / $4.25 | $2.00 |
| DeepSeek V4 FlashByteDance | 40.8 | 69.1 | 117.7 | $0.14 / $0.28 | $0.175 |
| Gemini 3.6 FlashGoogle | 40.3 | 69.2 | 188.5 | $0.75 / $3.75 | $1.50 |
| Gemini 3.5 FlashGoogle | 38 | No value | 230.7 | $1.50 / $9.00 | $3.375 |
| Gemini 3.1 ProGoogle | 36.7 | 68.8 | 123.2 | $2.00 / $12.00 | $4.50 |
| Kimi K2.6Novita | 35.8 | 61.8 | No value | $0.80 / $3.40 | $1.45 |
| Qwen3.8 27BCerebras | 33.8 | 58.2 | 51.7 | $0.99 / $1.49 | $1.115 |
| Grok 4.20xAI | 30 | No value | No value | $1.25 / $2.50 | $1.5625 |
| Nemotron-3 Ultra 550B A55BTogether AI | 29.3 | 49.3 | 157.9 | $0.60 / $3.60 | $1.35 |
| Grok 4.3xAI | 29.1 | No value | 120.5 | $1.25 / $2.50 | $1.5625 |
| Gemini 3.5 Flash LiteGoogle | 27.6 | 49.3 | 338.6 | $0.30 / $2.50 | $0.85 |
| Kimi K2.5Novita | 27.6 | 46.8 | No value | $0.60 / $3.00 | $1.20 |
| DeepSeek V3.2Novita | 25.3 | 44.2 | No value | $0.269 / $0.40 | $0.3018 |
| Muse Glimmer 30BTogether AI | 24.4 | 49 | 103.4 | $0.35 / $1.50 | $0.6375 |
| Kimi K2 0905Novita | 17.2 | No value | No value | $0.60 / $2.50 | $1.075 |
| Gemini 2.5 ProOpenRouter | 16.7 | 46.7 | No value | $1.25 / $10.00 | $3.4375 |
| GPT Oss 120BAkashML | 15.6 | 30.4 | 216.7 | $0.037 / $0.187 | $0.0745 |
| DeepSeek V3.1Novita | 14.5 | No value | No value | $0.27 / $1.00 | $0.4525 |
| GPT Oss 20BAkashML | 10.8 | 20.7 | 236.3 | $0.02 / $0.10 | $0.04 |
| Gemini 2.5 Flash LiteOpenRouter | 9.2 | No value | No value | $0.10 / $0.40 | $0.175 |
| Nemotron-3 Nano 30B A3BNovita | 8.6 | 14.4 | 235.5 | $0.05 / $0.20 | $0.0875 |
| Gemini 2.5 FlashOpenRouter | 8.2 | No value | No value | $0.30 / $2.50 | $0.85 |
| Qwen3 Next 80B A3B InstructNovita | 7.8 | No value | 183.4 | $0.15 / $1.50 | $0.4875 |
| Qwen 2.5 72B InstructNovita | 3.9 | No value | No value | $0.38 / $0.40 | $0.385 |
| Gemma 3 12B ITNovita | 1 | 5.8 | No value | $0.05 / $0.10 | $0.0625 |
Output speed
Output tokens per second once a response has started, the median Artificial Analysis measured. 24 models, fastest first.
- Gemini 3.8 Flash
- Gemini 3.5 Flash Lite
- Gemini 3.7 Flash
- Muse Spark 1.2
- GPT Oss 20B
- Nemotron-3 Nano 30B A3B
- Gemini 3.5 Flash
- Muse Spark 1.3
- GPT Oss 120B
- Gemini 3.6 Flash
- Qwen3 Next 80B A3B Instruct
- Nemotron-3 Ultra 550B A55B
- Gemini 3.1 Pro
- Grok 4.3
- DeepSeek V4 Flash
- Muse Glimmer 30B
- Qwen3.8 Flash
- DeepSeek V4 Pro
- Grok 4.6
- Claude Fable 5.1
- Grok 4.5
- Qwen3.8 27B
- Kimi K3
- Qwen3.8 Max
Source: Artificial Analysis, via OneInfer’s catalog. Fetched 21 October 2026, 22:02 UTC.
How it is measured: Artificial Analysis performance methodology
Show the data as a tableHide the table
| Model | Output speed, tokens/s |
|---|---|
| Gemini 3.8 Flash | 355.8 |
| Gemini 3.5 Flash Lite | 338.6 |
| Gemini 3.7 Flash | 324.6 |
| Muse Spark 1.2 | 262 |
| GPT Oss 20B | 236.3 |
| Nemotron-3 Nano 30B A3B | 235.5 |
| Gemini 3.5 Flash | 230.7 |
| Muse Spark 1.3 | 221 |
| GPT Oss 120B | 216.7 |
| Gemini 3.6 Flash | 188.5 |
| Qwen3 Next 80B A3B Instruct | 183.4 |
| Nemotron-3 Ultra 550B A55B | 157.9 |
| Gemini 3.1 Pro | 123.2 |
| Grok 4.3 | 120.5 |
| DeepSeek V4 Flash | 117.7 |
| Muse Glimmer 30B | 103.4 |
| Qwen3.8 Flash | 76.4 |
| DeepSeek V4 Pro | 62.1 |
| Grok 4.6 | 60.5 |
| Claude Fable 5.1 | 53.5 |
| Grok 4.5 | 52.6 |
| Qwen3.8 27B | 51.7 |
| Kimi K3 | 42.2 |
| Qwen3.8 Max | 38.1 |
Reading these charts
- Intelligence Index and Coding Index
- Artificial Analysis combines a set of evaluations into one score to assess capabilities “across reasoning, knowledge, maths and programming”. The Coding Index is its score from the coding evaluations. Higher is better.
- Output speed
- Measured by Artificial Analysis on the APIs it tests for each model, not on OneInfer. The speed you get depends on the provider that serves your request.
- Price
- OneInfer’s lowest live listing for the model, blended 3 parts input to 1 part output, like the 3:1 blended price in the Artificial Analysis API.
- Teal dots
- No model in the chart scores higher for the same or a lower price. A good place to start testing, not a verdict on your workload.
What the catalog holds
- 42 models in our price list have Artificial Analysis values in the catalog; 37 are shown. Left out: Gemini 3.1 Flash-Lite, Kimi K2 Instruct, Kimi K2.7 Code, Qwen3 235B A22B Instruct 2507, Qwen3 235B A22B Thinking 2507, because the catalog files the same values under more than one model, so the match cannot be checked.
- The speed of 13 models is left out: the catalog holds the placeholder it writes when no speed was measured, which cannot be told apart from a real value.
- The catalog holds no time to first token for any model right now, so there is no chart for it.
- The values come from the catalog’s public insights API,
GET api.oneinfer.ai/v1/models/insightsThe response does not say when they were copied from Artificial Analysis or which version of its index they belong to.
Based on data from Artificial Analysis. This analysis was generated by OneInfer and does not represent the views or endorsement of Artificial Analysis.
Live prices across model tiers
Input, output and cached-input prices per million tokens for a lean, a middle and a top model in each family, as our pricing API lists them right now. Where several providers serve a model, the lowest listing is shown.
Price per million output tokens, by tier
The lean, middle and top model of each family at its lowest live listing. Output tokens usually cost several times more than input tokens, so they drive most bills.
- Lean tier
- Middle tier
- Top tier
Anthropic
- Claude Haiku 5.5Lean tier
- Claude Sonnet 5.5Middle tier
- Claude Opus 5.5Top tier
OpenAI
- GPT-6 LunaLean tier
- GPT-6.1 SolMiddle tier
- GPT 6 AstraTop tier
- Gemini 3.5 Flash LiteLean tier
- Gemini 3.8 FlashMiddle tier
- Gemini 3.1 ProTop tier
Open-weight
- GPT Oss 120BLean tier
- DeepSeek V4 FlashMiddle tier
- DeepSeek V4 ProTop tier
Fetched 21 October 2026, 22:02 UTC from GET api.oneinfer.ai/v1/developer/get-model-pricing, which returns US cents; shown here in dollars per million tokens. The live list has 166 text model listings from 18 providers, and the calculator below includes all of them.
Show the data as a tableHide the table
| Model | Input | Output | Cached input |
|---|---|---|---|
| Anthropic | |||
| Claude Haiku 5.5Anthropic | $0.50 | $2.50 | $0.05 |
| Claude Sonnet 5.5Anthropic | $2.00 | $10.00 | $0.10 |
| Claude Opus 5.5Anthropic | $4.00 | $20.00 | $0.20 |
| OpenAI | |||
| GPT-6 LunaOpenAI | $0.10 | $0.50 | $0.01 |
| GPT-6.1 SolOpenAI | $2.00 | $10.00 | $0.10 |
| GPT 6 AstraOpenAI | $10.00 | $50.00 | $1.00 |
| Gemini 3.5 Flash LiteGoogle, lowest of 2 listings | $0.30 | $2.50 | $0.03 |
| Gemini 3.8 FlashGoogle, lowest of 2 listings | $0.75 | $3.75 | $0.075 |
| Gemini 3.1 ProGoogle, lowest of 2 listings | $2.00 | $12.00 | $0.20 |
| Open-weight | |||
| GPT Oss 120BAkashML, lowest of 4 listings | $0.037 | $0.187 | $0.037 |
| DeepSeek V4 FlashByteDance, lowest of 3 listings | $0.14 | $0.28 | $0.028 |
| DeepSeek V4 ProNovita, lowest of 3 listings | $1.60 | $3.20 | $0.135 |
One model, several providers, different prices
Open-weight models are often served by more than one provider. These are the widest gaps in the live list. Providers can also differ in speed, precision, context length and rate limits, so the lowest price is one input to a routing decision, not the whole decision.
Output price by provider, against the lowest listing
Each row is a model several providers list; the count is in brackets. Dots are their output prices as a multiple of the lowest one, so every row starts at 1x.
- Lowest listing
- Other providers
- GPT Oss 20B (3)4.5x
- DeepSeek V4 Flash (3)4.3x
- DeepSeek V3.1 (2)4.2x
- GPT Oss 120B (4)4.0x
- Qwen3.8 27B (3)2.0x
- Qwen3.6 27B (3)1.9x
- Qwen3.6 35B A3B (3)1.7x
- MiniMax M3 (3)1.5x
Gap = highest listed output price divided by the lowest, from the same API response, fetched 21 October 2026, 22:02 UTC.
Show the data as a tableHide the table
| Model | Lowest output price | Highest output price | Output price gap |
|---|---|---|---|
| GPT Oss 20B3 listings | $0.02 in, $0.10 outAkashML | $0.18 in, $0.45 outThinking Machines | 4.5x |
| DeepSeek V4 Flash3 listings | $0.14 in, $0.28 outByteDance | $0.30 in, $1.20 outDeepSeek | 4.3x |
| DeepSeek V3.12 listings | $0.27 in, $1.00 outNovita | $1.695 in, $4.215 outThinking Machines | 4.2x |
| GPT Oss 120B4 listings | $0.037 in, $0.187 outAkashML | $0.35 in, $0.75 outCerebras | 4.0x |
| Qwen3.8 27B3 listings | $0.99 in, $1.49 outCerebras | $0.42 in, $3.00 outNovita | 2.0x |
| Qwen3.6 27B3 listings | $0.60 in, $3.00 outGroq | $1.86 in, $5.595 outThinking Machines | 1.9x |
| Qwen3.6 35B A3B3 listings | $0.10 in, $0.90 outAkashML | $0.248 in, $1.485 outNovita | 1.7x |
| MiniMax M33 listings | $0.29 in, $1.18 outOneInfer | $0.45 in, $1.80 outMiniMax | 1.5x |
What routing saves, worked out on live prices
Routing lowers a bill only when a cheaper model can handle part of the traffic. Set your volume, the share of short, simple requests and the two models; the arithmetic is printed under the result.
Everything on Claude Sonnet 5.5
$400
per month
50% on Claude Haiku 5.5, the rest on Claude Sonnet 5.5
$250
per month
Routing costs $150 less a month under these assumptions, 37.5% of the frontier-only bill.
frontier only
100M x $2.00 + 20M x $10.00 = $400
lean only
100M x $0.50 + 20M x $2.50 = $100
routed = share x lean + rest x frontier
50% x $100 + 50% x $400 = $250
difference
$400 - $250 = $150 (37.5%)The saving is always the routed share times the cost gap between the two models. This compares list prices only: it assumes the lean model answers those requests well enough, which you should test on your own prompts first, and it leaves out cached-input discounts and anything billed outside per-token prices.
What we measure and how
We use the definitions public benchmarks use, so our numbers can be compared with theirs and rerun by anyone with an API key.
- Time to first token
- Time from sending a request to receiving the first streamed token, measured at the client. It includes network distance, queueing and prompt processing, so every result says where the client ran.
- Output speed
- Tokens per second once output has started: output tokens minus one, divided by the time from the first token to the end of the stream.
- End-to-end time
- Time from sending the request to the last token, for a stated prompt length and output length.
- Price per million tokens
- The list price for input, output and cached input tokens, read from our pricing API at the time shown. Input and output stay separate, because the mix is different for every workload.
- Workload cost
- Input tokens times the input price plus output tokens times the output price, for each model a workload uses. The calculator on this page prints every step.
- Kernel speedup
- Reference runtime divided by optimized runtime on the same GPU, inputs and precision, counted only for kernels that first match the reference output.
Every published result states
- The date and the time window of the runs
- The model id and the provider that served it
- Where the client ran, on what machine, against which endpoint
- Prompt length, output length and concurrency
- How many warm-up and measured runs
- The median, the 90th percentile and the range, never a single best run
- Failed requests, counted rather than dropped
- The script version and a hash of the script that wrote the file
The public methods we follow
- Artificial Analysis, performance methodology
Defines time to first token and output speed the way we use them. It tests endpoints eight times a day with about 1,000 and 10,000 input tokens (100,000 weekly) and reports the median of the past 72 hours, from a Google Cloud machine in us-central1-a.
- MLPerf Inference rules, MLCommons
A speed result counts only if the model also meets an accuracy target, runs last at least 600 seconds and the system is fully described. In its words, results that cannot be replicated are not valid results.
- vLLM benchmark suite (vllm bench serve)
Reports time to first token, time per output token, inter-token latency and end-to-end latency as mean, median and percentiles, with request and token throughput.
- KernelBench, Stanford (ICML 2025)
Checks each generated kernel against the PyTorch reference on 5 random inputs, times it over 100 trials after warm-up, and reports fast_p: the share of tasks that are correct and more than p times faster than the baseline.
- LLMPerf, Anyscale
Warns that API results vary with the time of day and with load, and may not match your workload. That is why we publish medians with ranges and the time window.
Sources read on 20 October 2026.
Latency and throughput
We have not published latency or throughput measurements yet. They will appear here, with the full setup, once a run has been reviewed. Until then this page shows no speed numbers of ours.
How a run works
- Streams each request from the OpenAI-compatible endpoint POST /v1/chat/completions and times the first token and the end of the stream at the client.
- Default workloads use about 1,000 and 10,000 input tokens with up to 400 output tokens, the same input sizes as the Artificial Analysis workloads.
- One warm-up request per model and workload, then 10 measured runs, interleaved across models so a slow minute lands on every model alike.
- A unique id opens every prompt, so provider prompt caches cannot shorten time to first token.
- Output tokens come from the usage count the API returns; when a provider leaves it out, the estimate is flagged in the file and on this page.
- The file keeps every run, failures included, and this page recomputes medians from the raw runs.
ONEINFER_API_KEY=... node scripts/benchmarks/run-latency.mjs \
--models anthropic/claude-haiku-5.5,openai/gpt-oss-120b \
--runs 10 \
--region "AWS us-east-1" --machine "EC2 c7i.large"
# writes src/data/benchmarks/latency-YYYY-MM-DD.json
# preview the plan without a key: --dry-runFor independent measurements of the providers behind these models, see Artificial Analysis.
Kernel optimization
We have not published kernel speedup results yet. When we do, each will follow the KernelBench method: correctness first, then speed against named baselines on named hardware, with the kernels alongside.
Correct first
Compare the optimized kernel with the PyTorch reference on several random inputs at a stated tolerance. A kernel that fails is reported as failed, not dropped.
Then fast
Time the reference and the kernel on the same GPU with warm-up and repeated trials, and report the speedup over PyTorch eager and over torch.compile.
Fully described
Name the GPU, driver, CUDA, PyTorch and Triton versions, the input shapes and the precision, and report the share of tasks that are correct and faster, not only the best case.
Rerunnable
Publish the generated kernels and the script with the numbers, so anyone with the same GPU can check them.
Kernel Forge is in preview in the OneInfer Console as SoL Optimizer. It runs on a GPU instance you choose, measures the open-source baseline (vLLM with default settings), and keeps a change only when repeated trials show a gain and a quality check passes.
Questions about a number?
If a performance or cost figure on oneinfer.ai is not backed on this page, tell us which one.