Benchmarks
Choose a path, the Mac Studio M5 Ultra or the Mac mini, then a workload. Measured on release 0.4.0 (release 0.4.1 runs the same engine), 2–5 November 2026, and how other engines compare on the same machine, weights and harness.
Choose a path
Your choice is remembered. Every chart and table says which machine it was measured on. Prices are Apple US list prices, November 2026.
Explorer
Every chart on the site, for the path you chose. Switch the workload, show or hide the engines and Macs, flip to the table, or copy a link to exactly what you are looking at. Each chart keeps its test, its date and where its raw data lives.
How much
How soon
How efficient
How good
Throughput
How much text does one Mac Studio write per second as the agent count grows?
Higher is better
| Agents | Flash-Next | Qwen3.8-27B | Splash 1.5 · 27B |
|---|---|---|---|
| 1 | 160 | 150 | 172 |
| 4 | 339 | 391 | 354 |
| 8 | 532 | 496 | 369 |
| 16 | 791 | 593 | 369 |
| 32 | 1,062 | 722 | – |
| 48 | 1,214 | 724 | – |
| 64 | 1,340 | 849 | – |
| 100 | 1,457 | 1,023 | – |
| 128 | 1,487 | 951 | – |
Total tok/s by number of agents (axis to scale), Mac Studio M5 Ultra, 512-token replies, 0 errors, 5 November 2026. Qwen3.8-27B 4-bit with the DFlash2 drafter, mean of 3 trials; Qwen3.8-Flash-Next with its MTP head, release 0.4.0, 128 snapshots, mean of 3 trials. The slowest agent's first word at 64 agents: 3.4–3.7 s (27B), 1.8–1.9 s (Flash-Next). Total tok/s, Mac Studio, Qwen3.8-27B 4-bit, 16k contexts, 512-token replies; inferd mean of 3 trials (5 November), Splash 1.5.0 one run (30 October). Splash 1.5 is faster for a single agent and levels off from 8 agents; it was not run beyond 16.
100 and 128 agents
Does the Mac Studio keep scaling when more agents than its 64 running slots all send a turn at the same moment?
Higher is better
| Model, agents | tok/s, mean | Round 1 | Round 2 | First token, median / max, round 1 (s) | Round 2 (s) | Correct | Generating at once |
|---|---|---|---|---|---|---|---|
| Flash-Next, 64 | 1,325.5 | 1,232.5 | 1,418.6 | 2.5 / 2.5 | 4.4 / 4.7 | 128 of 128 | 64 of 64 |
| Flash-Next, 100 | 1,457.0 | 1,358.6 | 1,555.5 | 3.3 / 4.1 | 6.6 / 8.2 | 200 of 200 | 100 of 100 |
| Flash-Next, 128 | 1,487.1 | 1,377.7 | 1,596.5 | 4.4 / 5.2 | 6.5 / 9.5 | 256 of 256 | 128 of 128 |
| 27B, 100 | 1,022.9 | 928.1 | 1,117.7 | 4.9 / 8.4 | 6.2 / 9.9 | 200 of 200 | 100 of 100 |
| 27B, 128 | 950.8 | 865.8 | 1,035.8 | 6.2 / 11.9 | 8.3 / 13.8 | 256 of 256 | 128 of 128 |
Qwen3.8-Flash-Next oQ4e with its MTP drafts and Qwen3.8-27B 4-bit with the DFlash2 drafter, the release 0.4.0 engine, Mac Studio M5 Ultra 256 GB, 5 November 2026. Every agent has its own 16k context; all send a follow-up turn at the same moment, 512 tokens out each. Each answer starts with the sum of two numbers that differ for every agent and every round, so a reply that came from another request, was cut short or garbled would be wrong: every one was right, and every agent was generating at the same moment in every round. The line to 64 is the 3-trial mean from the throughput chart; the three points at 64, 100 and 128 are a separate run of 2 rounds with a different harness, which is why the two Flash-Next 64-agent points differ by about 1%. The 27B gives 1,023 tok/s at 100 agents and 951 at 128. Flash-Next at 128 was run three times: 1,417 / 1,485 / 1,487 tok/s (the chart shows the run that also measured power).
Mac mini throughput
How much text does a Mac mini write per second as the agent count grows?
Higher is better
| Agents | mini M5 Pro | mini M6 | mini M4 Pro |
|---|---|---|---|
| 1 | 47 | 28 | 24 |
| 4 | 113 | 65 | 50 |
| 8 | 148 | 66 | 56 |
| 16 | 168 | 68 | 60 |
Total tok/s, Qwen3.8-27B 4-bit with the DFlash2 drafter, 16k contexts, 512-token replies, Mac minis, release 0.2.0, 4 November 2026. The M6 has 32 GB; the M5 Pro and M4 Pro have 64 GB. The 27B fits on a 32 GB mini; Qwen3.8-Flash-Next does not.
9B on every mini
How much text does each Mac mini write per second with 16 agents on a 9B model?
Higher is better
Total tok/s, Qwen3.5-9B 4-bit, 16 agents on 16k contexts, Mac minis, release 0.2.0, 4 November 2026. Where a range was measured the bars use inferd's lowest and TensorFold's highest figure: M6 158–160 against 23, M5 Pro 287–295 against 40–46. 8-bit on the M6: 115 against 8.7.
Launch from a saved context
How soon can agents start from a saved context instead of each reading the document?
Lower is better
Mac mini M6 32 GB, Qwen3.8-27B, 8 agents forked from a long context: 75.2 s when the context is read cold, 2.36 s when they launch from the saved context (RESULTS (o)).
Across Macs
How does a Mac mini compare with the Mac Studio on the same model and the same kind of work?
Higher is better
| Agents | Studio M5 Ultra | mini M5 Pro | mini M6 | mini M4 Pro |
|---|---|---|---|---|
| 1 | 150 | 47 | 28 | 24 |
| 4 | 391 | 113 | 65 | 50 |
| 8 | 496 | 148 | 66 | 56 |
| 16 | 593 | 168 | 68 | 60 |
| 32 | 722 | – | – | – |
| 48 | 724 | – | – | – |
| 64 | 849 | – | – | – |
Total tok/s, Qwen3.8-27B 4-bit with the DFlash2 drafter, 16k contexts, 512-token replies. Mac Studio M5 Ultra: release 0.3.0, mean of 3 trials, 5 November 2026. Mac minis: release 0.2.0, 4 November 2026, run up to 16 agents. The releases differ, so read the gap as the scale of each machine, not as a precise ratio; the minis are not a smaller Studio, they are smaller machines, and each does hard work at its own tier.
Long context
With a 64,000-token context per agent, how long does the slowest agent wait for its first word?
Lower is better
| Agents | inferd | Splash 1.5 |
|---|---|---|
| 1 | 0.2 | 0.2 |
| 4 | 0.5 | 1.1 |
| 8 | 0.8 | 8.0 |
| 16 | 1.6 | 22.2 |
| 32 | 3.4 | 48.9 |
Time to the last agent's first token, seconds. Agents taking turns on 64k contexts, Mac Studio. inferd 5 November (2 trials, mean); Splash 1.5.0 30 October (one run).
Start together
If N agents each send a long prompt at the same moment, how long until the last one finishes?
Lower is better
| Agents released together | inferd · 27B | TensorFold 1.2.2 · 27B | inferd · Flash-Next | TensorFold 1.2.2 · Flash-Next |
|---|---|---|---|---|
| 1 | 29.8 | 32.1 | 39.4 | 29.9 |
| 2 | 53.4 | 53.9 | 57.2 | 50.3 |
| 3 | 61.5 | 71.4 | 73.4 | 66.6 |
| 4 | 75.7 | 91.3 | 89.2 | 86.7 |
| 5 | 93.3 | 113.1 | 105.9 | 104.6 |
| 6 | 101.4 | 133.3 | 118.6 | 125.1 |
| 7 | 116.7 | 153.6 | 132.7 | 145.2 |
| 8 | 125.6 | 177.6 | 137.7 | 166.1 |
Seconds until the last agent finishes. N agents released together, 14,336-token prompts, 6,400 tokens out, Qwen3.8-27B, same drafter weights, mean of 3 trials. TensorFold 1.2.2 (3 November); 1.2.1 measured the same within run-to-run noise.
Released at once
On the same Mac Studio, who finishes a batch of long prompts first, and does it still work at 64?
Higher is better
Higher is better
Higher is better
Higher is better
Random arrivals
When 16 agents turn up a few seconds apart with prompts of every size, how long does the whole job take?
Lower is better
16 agents arriving about 3 s apart with 2k–16k prompts and short replies, Qwen3.8-27B. TensorFold 1.2.2 in its default order; its new shortest-first order measured slower (122.9 s). On Flash-Next the two engines are level here (57.2 vs 57.3 s).
Resume
After other work has filled memory, how long until a returning agent gets its first word?
Lower is better
Seconds until an agent's first token after other work filled memory. Without the agent API the agent re-reads its whole conversation; with it, inferd resumes the saved session from memory or SSD. Qwen3.8-27B 4-bit + DFlash2 on Mac minis. The log scale shows the short bars clearly.
vs DGX Spark
Both hold 256 GB. Which serves Flash-Next to a handful of agents released together?
Higher is better
| Agents | Mac Studio, inferd | TensorFold, 2 DGX Sparks |
|---|---|---|
| 1 | 162 | 92 |
| 4 | 261 | 176 |
| 8 | 341 | 240 |
Qwen3.8-Flash-Next, agents released together, same harness, single trial each (2 November 2026): inferd on one Mac Studio against TensorFold's dual-Spark recipe (tensor parallel over two NVIDIA DGX Sparks, which hold the same 256 GB as one Studio). Measured in our lab; the Spark runs are not part of the inferd repository's results. A DGX Spark version of inferd is coming soon.
Energy
How much energy does the whole Mac Studio use per token as the agent count grows?
Higher is better
Output rises with the agent count while the power at the wall stays at about 300 W. Idle at the wall before each run: 13.5 W (64), 14.1 W (100), 15.0 W (128).
| Agents | Round | tok/s | Wall W | macOS total W | GPU W | CPU W | DRAM W | tok/J | J per 1,000 tokens (derived) |
|---|---|---|---|---|---|---|---|---|---|
| 64 | 1 | 1,232.5 | 295.9 | 276.2 | 120.0 | 0.9 | 24.1 | 4.2 | 238 |
| 64 | 2 | 1,418.6 | 300.5 | 264.8 | 122.4 | 1.7 | 21.1 | 4.7 | 213 |
| 64 | mean | 1,325.5 | 298.2 | 270.5 | 121.2 | – | – | 4.4 | 227 |
| 100 | 1 | 1,358.6 | 303.4 | 282.7 | 125.9 | 0.9 | 24.0 | 4.5 | 222 |
| 100 | 2 | 1,555.5 | 297.0 | 272.3 | 129.3 | 1.6 | 20.5 | 5.2 | 192 |
| 100 | mean | 1,457.0 | 300.2 | 277.5 | 127.6 | – | – | 4.9 | 204 |
| 128 | 1 | 1,377.7 | 297.6 | 276.9 | 122.1 | 3.0 | 22.7 | 4.6 | 217 |
| 128 | 2 | 1,596.5 | 287.6 | 269.1 | 124.4 | 2.8 | 19.6 | 5.5 | 182 |
| 128 | mean | 1,487.1 | 292.6 | 273.0 | 123.2 | – | – | 5.1 | 196 |
The same runs as the previous chart: Qwen3.8-Flash-Next oQ4e with its MTP drafts, release 0.4.0, Mac Studio M5 Ultra 256 GB, 5 November 2026, every agent on its own 16k context, 512 tokens out each, 2 rounds after priming. Wall power is the whole Mac Studio at the socket, read once a second from a smart plug, so it includes the fans, the SSD and the Mac's idle draw; macOS's own counters put the chip's GPU at about 121-128 W. tok/J is tok/s divided by watts at the wall (tokens per joule). One run of 2 rounds per agent count; run-to-run spread is a few percent.
Quality
Does the speed cost accuracy? One bounded check among several, not proof of equivalence.
Higher is better
Percent correct, effort medium, Mac Studio, inferd a4ee879 (32k budget, capped items re-asked at 128k). Published runs on other engines: 27B 31.8–33.5% (effort low), Flash-Next 34.7–43.9%.* Benchmark: github.com/incept5-ai/hle-lite
The tables
What each Mac did on release 0.4.0, then the same work next to other engines, with where they lead.
M6, M5 Pro and M4 Pro, side by side
| Mac mini | What it does | inferd |
|---|---|---|
| M6, 32 GB | Qwen3.8-27B + drafter, 8 agents taking turns on 16k contexts, total tok/s (opt-in 4-bit KV) | 95 |
| M6, 32 GB | Qwen3.5-9B, 16 agents taking turns on 16k contexts, total tok/s | 164 (8-bit: 117) |
| M6, 32 GB | 27B agents launched from a saved context / resumed after other work: first token | 2.3 s / 0.4 s |
| M5 Pro, 64 GB | Qwen3.8-27B + drafter, 16 agents taking turns on 16k contexts, total tok/s | 162–164 |
| M5 Pro, 64 GB | 35B, 32 agents taking turns on 4k / 16k contexts, total tok/s | 610 / 467 |
| M5 Pro, 64 GB | 27B on 32k contexts, 16 agents: total tok/s, the slowest agent's first token (8-bit / 4-bit KV) | 142 / 153, 7.5 / 5.8 s |
| M5 Pro, 64 GB | Agents resumed after unrelated load (35B / 27B): first token | 0.14 s / 0.25 s |
| M4 Pro, 64 GB | 27B, one agent, 8k prompt: first token (the Neural Engine helps on M4) | 42.8 s |
A turn is one agent step: read its context plus new input, then reply. tok/s is tokens per second: a token is about three-quarters of a word. Flash-Next does not fit on a mini: for the biggest models, step up to the Studio.
256 GB · $10,799
| What it does | inferd |
|---|---|
| Qwen3.8-27B, 64 agents taking turns on 16k contexts: total tok/s, slowest first token (32 agents: 722 tok/s) | 849, 3.4–3.7 s |
| Qwen3.8-Flash-Next, 64 agents on 16k contexts: total tok/s, slowest first token (48 agents: 1,214 tok/s) | 1,331–1,346, 1.8–1.9 s |
| Qwen3.6-35B-A3B (mixture of experts), 32 agents on 16k contexts: total tok/s, slowest first token | 1,079, 1.7 s |
| 32 agents on 64k contexts: the last agent's first token | 3.3–3.4 s |
| One agent saves a 33.5k-token context, 31 more start from it: median first token / all 31 done | 0.72 s / 15.3 s |
| 32 agents forked from a 25k context, relaunched after 1.4M tokens of unrelated work: first token | 1.03 s |
Mac Studio M5 Ultra, 256 GB, measured 4–5 November 2026.
M6, M5 Pro and M4 Pro, next to other engines
| Mac mini | Workload | inferd | Other engine |
|---|---|---|---|
| M6, 32 GB | 9B, 16 agents on 16k, total tok/s | 164 (8-bit 117) | TensorFold 1.2.2: 23 (8-bit 8.7) |
| M6, 32 GB | 27B, 8 / 16 agents on 16k, total tok/s (opt-in 4-bit KV) | 95 / 85 | Splash 1.5: 86 / 89; TensorFold 1.2.2 cannot load the 27B on 32 GB |
| M5 Pro, 64 GB | 27B, 16 agents on 32k, total tok/s, slowest first token (opt-in 4-bit KV) | 153, 5.8 s | Splash 1.5: 146, 44.3 s |
| M5 Pro, 64 GB | 9B agent fleet, 32 agents, turns per minute (4-bit / 8-bit) | 57.4 / 46.9 | TensorFold 1.2.2: 7.7 / 3.0 |
| M5 Pro, 64 GB | 16 forked agents relaunched / 8 resumed after unrelated load: first token | 2.1 s / 0.25 s | TensorFold 1.2.2 (disk cache): 195 s / 15.4 s |
| M4 Pro, 64 GB | 27B, one agent, 8k prompt: first token | 42.8 s | mlx_lm: ~63 s |
Where others lead on the minis. Splash is faster for one agent and for 4–8 agents taking turns on the M5 Pro and M6 (for example the 35B at 16k with 4 agents, 315 vs 237 tok/s on the M5 Pro), and for the 35B on the 32 GB M6 at 16k. Splash reads one cold prompt sooner on the M5 Pro and M6. On M1–M4 GPUs leave the drafter off for the 35B from 4 agents (--draft none).
256 GB · $10,799 · Mac Studio, next to other engines
| Workload | inferd | Other engine |
|---|---|---|
| 27B, 32 agents on 16k contexts, total tok/s | 722 | Splash 1.5: levels off at ~369 |
| 27B agent fleet on 8k, 8 / 32 / 64 in flight, turns per minute | 80 / 89 / 95 | Splash 1.5: 49 / 50 / 50 (32 of 128 turns refused at 64) |
| 35B, 32 agents on 16k, total tok/s | 1,079 | Splash 1.5: 598 |
| 32 agents on 64k: the last agent's first token | 3.3–3.4 s | Splash 1.5: 48.9 s |
| 8 Qwen3.8-27B agents start at once (14k prompts, 6,400 tokens out): all done | 125.6 s | TensorFold 1.2.2: 177.6 s |
| 16 agents arriving at random, 2k–16k prompts: all done | 78.7 s | TensorFold 1.2.2: 116.8 s |
Where others lead on the Studio. One agent on its own: Splash, TensorFold and mlx-serve decode a single stream faster. Flash-Next with 1–4 agents starting together: TensorFold 1.2.2 finishes sooner (level at 5); with 16 arriving at random the two are level on total time and TensorFold's p95 first token is lower (9.2 vs 13.7 s). MacBook Pro M3 Max (external tester): 16 agents at 32k, the last agent's first token 28.4 s vs Splash 1.5's 113–115 s.
Quality
Checked on our published logits, coding and HLE Lite tests (HLE Lite: 173 text-only multiple-choice items, not the full HLE leaderboard). These are bounded checks, not proof of equivalence; results and limits are in the repository.
Writes working code
43.7/48Coding test*, level with Splash 1.5.0 (41.8, within run-to-run noise; 2 November).* Benchmark: github.com/incept5-ai/languages-benchmarkValid JSON, first time
100/100Strict-JSON replies valid on the first attempt, across five sets of 20 under mixed load.Same answers as the reference
top-1Logits match the mlx_lm reference: the same top token, cosine 0.999 or higher, from 1- to 1,536-row chunks.The HLE Lite chart is in the explorer.
Fair's fair
inferd is not ahead everywhere, and it stands on excellent work. These are the engines we measure ourselves against, and where they lead.
Open models
Macs
macOS 27; the macOS 26 build is in beta (it passed the full release gate on two macOS 26 Macs so far). M1 and M2 Macs are untested. Coming soon: a version of inferd for the NVIDIA DGX Spark.
Keep going