The convenience of cloud. The benefits of local.
Flat-rate dedicated capacity keeps costs flat while usage scales and offers robust IP & data protection.
AI is great until it hits production.
A workflow, a feature, or a demo can work. Press go at production scale and throughput, plus an error rate that compounds and you blow past the budget, throttle the product, and collapse the experience.
Budgets balloon.
Token meters turn production volume into a bill that scales out of control.
Throughput throttles.
The product slows under real traffic, and the experience the demo promised does not hold.
A 5–10% error rate compounds.
A few bad tool calls stack until the workflow fails and the experience collapses.
Built for Production
Dedicated capacity
Dedicated capacity holds the budget. The tool-calling runtime holds the error rate. The serving runtime holds the throughput.
The bill stays flat. Production can run as hard as the machine allows.
Cost vs usage
Usage-based pricing
Dedicated capacity
Usage-based pricing climbs. Dedicated capacity stays flat.
Reserved for your team. One flat rate.
Dedicated capacity
Starts at $444/mo. Hosted in our facility.
Hub license
Vaults, sub-agents, and API on hardware you own.
Throughput
The #1 inference engine on Mac.
Fastest decode in every test we ran, against LM Studio, Ollama, and oMLX. Apple M5 Max, 128 GB · October 2026.
- First token, 33k prompt
- 23×
- ~12.4s vs Ollama’s ~285s · oMLX ~12.9s
- Editing a file
- 3.7×
- 268.1 vs 71.8 tok/s · 1.5× oMLX
- Short coding ask
- 133 tok/s
- 2.0× Ollama · +5% oMLX
125B mixture of experts · tuned for Apple Silicon
A 125B model at over 130 tokens a second.
Flash Next is a brand-new architecture, and Midium runs it on an engine path tuned specifically for it, built around the model’s own multi-token prediction (MTP). Same model, a lot more speed, and long prompts start answering in seconds, not minutes.
- 133 tok/s on a short coding ask: 2× Ollama, 5% ahead of oMLX.
- 3.7× Ollama’s speed, and 1.5× oMLX’s, when the answer repeats the prompt, like returning an edited file.
- 23× sooner to a first token on a 33k-token prompt: about 12.4s against 285s.
| Test | Midium · MTP on | oMLX · MTP on | Ollama · MTP on | Midium · MTP off |
|---|---|---|---|---|
| Decode | ||||
| Short coding ask42-token prompt | 132.8tok/sbest 141.8↑ 5% oMLX · 2.0× Ollama · 1.7× MTP off | 126.6best 132.4 | 67.8best 71.7 | 78.3 |
| Write a new moduleAfter a 33k-token codebase | 96.0tok/sbest 103.6↑ 11% oMLX · ↑ 84% Ollama · 1.4× MTP off | 86.3best 93.0 | 52.2best 54.4 | 70.0 |
| Edit a file6k-token file, one change | 268.1tok/sbest 278.2↑ 52% oMLX · 3.7× Ollama · 3.7× MTP off | 176.6best 182.9 | 71.8best 73.7 | 72.6 |
| Same edit, as a tool callWhole file saved through write_file · valid on all three | 239.2tok/sbest 240.8↑ 39% oMLX · 3.2× Ollama · 3.6× MTP off | 171.5best 176.1 | 74.6best 75.5 | 66.0 |
| Prompt processing | ||||
| Prefill, 6kFirst token ~2.3s · ~2.5s · ~31s | 2,450tok/sbest 2,636≈ oMLX · 12× Ollama | 2,476best 2,592 | 200best 214 | 2,596 |
| Prefill, 33kFirst token ~12.4s · ~12.9s · ~285s | 2,651tok/sbest 2,697↑ 4% oMLX · 23× Ollama | 2,554best 2,650 | 115best 119 | 2,583 |
Each figure is the mean of 9 runs (3 rounds × 3 seeds), “best” the fastest run; MTP off is the mean of 3 runs.
All three runtimes draft with the model’s own MTP head, and oMLX runs the exact 4-bit build Midium does. LM Studio couldn’t load this model, so it has no column (see the methodology). Setup and measurement details are in the methodology below.
Methodology
- Machine
- The same MacBook Pro, Apple M5 Max, 128 GB with fans at full speed, the chip cooled to 40 °C or lower (held for 5 seconds) before every job. Midium on MLX 0.32.3, oMLX 0.7.1.dev1, Ollama 0.40.0. Ollama needed the GPU memory limit raised to ~120 GB to avoid running out of memory; Midium runs at the default.
- Weights
- Midium and oMLX run the same oQ4e MLX build (99 GB), Ollama its official NVFP4 MLX build (104 GB). All include the model’s multi-token prediction (MTP) head, and all three runtimes draft with it.
- Requests
- Temp 1.0, top-p 0.95, top-k 20, thinking off, streaming, 3 seeds per job, 3 rounds per runtime. Every prompt ran cold: a unique prefix on each Midium and oMLX request, a model reload before each Ollama run, and one unmeasured warm-up after every load. MTP off is Midium on the same weights, one token per step.
- Measuring
- Decode is each runtime’s own reported rate after the first token. Prefill is prompt tokens over the time to a first token (Ollama: its own reported prompt-processing rate). Tool calls: the whole 6k-token file saved through write_file — every call valid on all three, with exactly the one change.
Measured October 13, 2026 · Qwen3.8 Flash Next · Apple M5 Max, 128 GB, cooled to 40 °C before every job · temp 1.0 · thinking off · mean of 9 runs, best run below.
Every Midium number here comes from Midium’s own inference engine, not a wrapper around llama.cpp, Ollama, LM Studio or another app’s server. We build it directly on Apple’s MLX and write the parts that matter ourselves: the Metal kernels, the scheduler, multi-token prediction, prompt caching, a tool-call grammar that keeps calls valid, and a guard that catches looping, runaway output before it wastes your time. It’s tuned for three things: speed, tool calls you can rely on, and stopping runaway generations.
Batching
Requests are batched through the runtime.
Model pooling
Several instances sit behind one API and take work round-robin.
Loop detection
A watchdog detects an endless-loop hallucination, kills the model, and restarts it so throughput keeps moving.
Error rate
A small miss does not get to compound.
A 5–10% error rate is how a working agent falls apart in production. Each bad tool call is another retry, and the misses stack until the experience collapses. We host the inference those agents call. The runtime is tuned for tool calling on Apple Silicon, on the same open weights a stock server would run. Gemma 4 26B A4B, 4-bit.
90.2%
Tool call accuracy on a 26B model at 4-bit. Ollama’s best is 87.4%. Parallel tool calls take a 13.5-point lead.
| Test | Midium | Ollama |
|---|---|---|
| ThinkingTool call accuracy | 90.2%↑ 2.8 Ollama thinking↑ 5.2 Ollama no-think | 87.4% |
| No-thinkSame test, thinking off | 89.7%↑ 2.3 Ollama thinking↑ 4.7 Ollama no-think | 85.0% |
| Parallel tool callsThe most difficult tool-calling test | No-think83.0% ↑ 5 Ollama thinking↑ 13.5 Ollama no-think | Thinking78.0% No-think69.5% |
Measured July 23, 2026 · Midium vs Ollama · Gemma 4 26B A4B, matched 4-bit · Apple Silicon, 128 GB · BFCL v4 official checker · 1,000 cases · temp 0.
Methodology: See the proof.
Improve efficiency
Tools For The Team.
Keep agents in sync, up to date, and efficient with vaults and offload usage with free coding sub agents.
Vaults
Persistent, Shared Agent Memory
Vaults are shared and scoped memory for engineers and their coding agents. Cursor, Claude Code, Codex, and your own tools read them. Agents write context back as they work, so a session spends less time reorienting, and each team member's agent stays in sync with what the others are doing.
- Your personal Vault is free in Midium Desktop. Shared Vaults come with dedicated capacity or a Hub license.
- Cloud context is not used for training. Local Vaults stay on the device.
Sub-agents
Maximize Output on Your Plan
Midium Desktop exposes local models over MCP to Cursor, Claude Code, and Codex. Runs on 16 GB. 48 GB recommended for the best models. Move at least 60M tokens a month off the plan you pay now, so the team can do more development or drop a subscription tier without cutting code throughput. Local inference and your personal Vault stay on your machine by default. Connectors only send what you point them at.
Start here
Start on tokens. Scale into Dedicated Capacity
Use our cloud on competitive token-based pricing with $5 free to start. Switch to Dedicated Capacity when you hit production scale.
Download the free Desktop App on any M-series Mac to use local AI APIs, utilize sub-agents with your coding agents, and get a personal Vault.
| Model | Tier | Cloud input / M | Cloud output / M |
|---|---|---|---|
| Gemma 4 E2B | Lite | $0.024 | $0.048 |
| Gemma 4 26B A4B | Balanced | $0.13 | $0.40 |
| Gemma 4 31BDedicated capacity | Balanced | — | — |
| Laguna-XS-2.1-6bit | Balanced | $0.06 | $0.12 |
| Qwen3.8 27BDedicated capacity | Balanced | — | — |
| GLM 5.3 FlashDedicated capacity | Frontier | — | — |
| Inkling SmallDedicated capacity | Frontier | — | — |
| Laguna S 2.1Dedicated capacity | Frontier | — | — |
| Qwen3.8 Flash NextDedicated capacity | Frontier | — | — |
| Midium Decider 4B | System One | $0.032 | $0.00 |
| BGE-M3 | Embeddings | $0.012 | $0.036 |
Shared models are billed per million tokens. Dedicated capacity is a flat rate, sized by throughput, team, and agent capacity.
