Private AI for teams of agents
From a Mac mini at about $1,000 to a Mac Studio at up to $20,000, inferd runs all your agents at once, on your own machine. Your data stays put, and there is no bill per word.
What one Mac sustains
128 agents · · slowest first word
W at the wall tokens per joule W idle
Energy
Measured at the wall with a smart plug, so it counts the fans, the SSD and the Mac's idle draw. Every answer was correct. Other engines and other Macs have not been measured this way yet.
Drop-in
inferd speaks the OpenAI API, so agent frameworks and apps that already talk to OpenAI connect by changing the base URL. The agent features are one extra field on the same request.
Any OpenAI client
# pip install openai
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8300/v1", api_key="not-needed")
reply = client.chat.completions.create(
model="qwen3.8-27b",
messages=[{"role": "user", "content": "Hello"}],
)
print(reply.choices[0].message.content)Only the base URL, the key and the model name differ from a call to OpenAI.
One field on a normal OpenAI request
POST /v1/contexts # save + pin a context once POST /v1/chat/completions { "model": "qwen3.8-27b", "messages": [ ... ], "inferd": { "context": "brief-42", # start from it "session": "agent-7", # keep my turn "effort": "low", # per agent "max_context": 32768, "priority": 1 } } # one-off work that leaves no trace: "inferd": { "cache": "drop" } GET /v1/cache # cached, pinned, parked, free
Every reply reports cached_tokens, so a framework can check its caching strategy instead of guessing.
Beyond tokens per second
Agents reread the same material and wait on tools. These three measurements are about that, each linked to its chart. All on the Mac Studio M5 Ultra.
What it runs
Nine scenarios on a Mac Studio, from auditing a 40,000-token report with 32 agents to tracing a money-mule ring. Nothing leaves the machine.
32 auditors, one 40,000-token report
31 agents writing at once; 99% of 6.5 M input tokens came from shared memory.
→32 analysts, one market wrap, about two minutes
31 analysts at once: 31 charts and a wrap.
→20 alerts, one hidden attack
10 of 10 on the answer key, traps avoided, with 11 investigators at once.
→Try it
From an unpacked release: install, start, ask. The download and checksum, and the macOS 26 build, are on the Get started page.
Sets up the Python front end and downloads Qwen3.8-27B with its draft model. About 17 GB.
./install.sh --model 27bThe engine and its OpenAI-compatible server start on port 8300, sized for your Mac.
./run.sh start --model 27bThe same request every client sends.
curl localhost:8300/v1/chat/completions -H 'Content-Type: application/json' \ -d '{"model":"qwen3.8-27b","messages":[{"role":"user","content":"Hello"}]}'Open source
Apache-2.0. The code, the benchmark harnesses and every raw result are in the repository. Tell us your models, your Mac and your workload and we will say where inferd fits.