Lite
Gemma 4 E2B
Fast and efficient agents built for speed and throughput. Best for quick answers and simple workflows.
model: "lite"
AI API
The convenience of cloud. The benefits of local.
Purpose built for AI and Agents in production software and workflows.
AI is great until it hits production.
A demo can work. Put the same agent in front of real traffic and the misses compound, the model names change out from under you, and every slow call stalls the next step.
A 5–10% error rate compounds.
A few bad tool calls stack until the workflow fails and the experience collapses.
Every model swap is a migration.
New weights ship constantly. Each one means a new name, a new eval, and new code.
Slow calls stall the loop.
Every agent step waits on the model, so latency multiplies across the run.
The solution
Accurate, convenient, and fast.
Accuracy
Agents are 5–10% more accurate and reliable with Midium.
AI is a probability engine, which means it is probably right most of the time. Midium mixes deterministic logic with these models to improve tool calling by up to 13% compared to other leading providers.
90.2%
Tool call accuracy on a 26B model at 4-bit. Ollama’s best is 87.4%. Parallel tool calls take a 13.5-point lead.
Methodology: See the proof.
Convenience
White glove model selection done for you.
In the API, simply put "lite", "balanced", or "frontier" and our API will route to whichever model we have selected to power that tier. No maintenance headache, no open-weight model expertise required. Select any model you want on our dedicated capacity tier.
Balanced
Gemma 4 26B
Well rounded agent balancing speed, cost and intelligence. Best for most use cases.
model: "balanced"
Frontier
Inkling Small
The most intelligent models at a higher cost. Best for complex use cases demanding the best results.
model: "frontier"
Embeddings
Search and retrieval. The current embedding model stays behind this name.
model: "midium"
System One
Decider 4B
Choice, score, and noul. The current System One model stays behind this name.
model: "midium"
See the API. Chat uses lite, balanced, frontier. Embeddings and System One use midium.
It is an OpenAI-compatible API. Embeddings and System One stay on their usual calls, with the model set to midium. When we change the weights, you do not change production.
Speed
Built for production throughput with tens of thousands of monthly active users.
Midium has some of the fastest speeds of any provider on OpenRouter, and is the #1 inference engine for speed and throughput on Mac. The highlights are the run. Open a table for the full benchmark.
- Short coding ask
- 133 tok/s
- 2.0× Ollama · +5% oMLX
- First token, 33k prompt
- 23×
- ~12.4s vs Ollama’s ~285s · oMLX ~12.9s
- Editing a file
- 3.7×
- 268.1 vs 71.8 tok/s · 1.5× oMLX
125B mixture of experts · tuned for Apple Silicon
A 125B model at over 130 tokens a second.
Flash Next is a brand-new architecture, and Midium runs it on an engine path tuned specifically for it, built around the model’s own multi-token prediction (MTP). Same model, a lot more speed, and long prompts start answering in seconds, not minutes.
- 133 tok/s on a short coding ask: 2× Ollama, 5% ahead of oMLX.
- 3.7× Ollama’s speed, and 1.5× oMLX’s, when the answer repeats the prompt, like returning an edited file.
- 23× sooner to a first token on a 33k-token prompt: about 12.4s against 285s.
| Test | Midium · MTP on | oMLX · MTP on | Ollama · MTP on | Midium · MTP off |
|---|---|---|---|---|
| Decode | ||||
| Short coding ask42-token prompt | 132.8tok/sbest 141.8↑ 5% oMLX · 2.0× Ollama · 1.7× MTP off | 126.6best 132.4 | 67.8best 71.7 | 78.3 |
| Write a new moduleAfter a 33k-token codebase | 96.0tok/sbest 103.6↑ 11% oMLX · ↑ 84% Ollama · 1.4× MTP off | 86.3best 93.0 | 52.2best 54.4 | 70.0 |
| Edit a file6k-token file, one change | 268.1tok/sbest 278.2↑ 52% oMLX · 3.7× Ollama · 3.7× MTP off | 176.6best 182.9 | 71.8best 73.7 | 72.6 |
| Same edit, as a tool callWhole file saved through write_file | 239.2tok/sbest 240.8↑ 39% oMLX · 3.2× Ollama · 3.6× MTP off | 171.5best 176.1 | 74.6best 75.5 | 66.0 |
| Prompt processing | ||||
| Prefill, 6kFirst token ~2.3s · ~2.5s · ~31s | 2,450tok/sbest 2,636≈ oMLX · 12× Ollama | 2,476best 2,592 | 200best 214 | 2,596 |
| Prefill, 33kFirst token ~12.4s · ~12.9s · ~285s | 2,651tok/sbest 2,697↑ 4% oMLX · 23× Ollama | 2,554best 2,650 | 115best 119 | 2,583 |
Each figure is the mean of 9 runs (3 rounds × 3 seeds), “best” the fastest run; MTP off is the mean of 3 runs.
All three runtimes draft with the model’s own MTP head. Ollama runs a different 4-bit build than Midium and oMLX because neither side can load the other’s: Ollama can’t load the build Midium runs, and Midium can’t load Ollama’s, so each is measured on the build it supports. LM Studio couldn’t load this model, so it has no column (see the methodology). Setup and measurement details are in the methodology below.
Methodology
- Machine
- The same MacBook Pro, Apple M5 Max, 128 GB with fans at full speed, the chip cooled to 40 °C or lower (held for 5 seconds) before every job. Midium on MLX 0.32.3, oMLX 0.7.1.dev1, Ollama 0.40.0. Ollama needed the GPU memory limit raised to ~120 GB to avoid running out of memory; Midium runs at the default.
- Weights
- Midium and oMLX run the same oQ4e MLX build (99 GB; oMLX needs a companion patch to load it), Ollama its official NVFP4 MLX build (104 GB). They can’t share one: Ollama can’t load the oQ4e build, and Midium can’t load Ollama’s. All include the model’s multi-token prediction (MTP) head, and all three runtimes draft with it.
- Requests
- Temp 1.0, top-p 0.95, top-k 20, thinking off, streaming, 3 seeds per job, 3 rounds per runtime. Every prompt ran cold: a unique prefix on each Midium and oMLX request, a model reload before each Ollama run, and one unmeasured warm-up after every load. MTP off is Midium on the same weights, one token per step.
- Measuring
- Decode is each runtime’s own reported rate after the first token. Prefill is prompt tokens over the time to a first token (Ollama: its own reported prompt-processing rate). Tool calls: the whole 6k-token file saved through write_file.
Measured October 13, 2026 · Qwen3.8 Flash Next · Apple M5 Max, 128 GB, cooled to 40 °C before every job · temp 1.0 · thinking off · mean of 9 runs, best run below.
Every Midium number here comes from Midium’s own inference engine, not a wrapper around llama.cpp, Ollama, LM Studio or another app’s server. We build it ourselves on Apple’s MLX and tune it model by model for three things: speed, tool calls you can rely on, and catching runaway output before it wastes your time.
Batching
Requests are batched through the runtime.
Model pooling
Several instances sit behind one API and take work round-robin.
Loop detection
A watchdog detects an endless-loop hallucination, kills the model, and restarts it so throughput keeps moving.
Privacy and security
Your prompts stay yours.
Prompts are not retained.
Prompts and generated responses are not retained. Midium does not train on them.
Vaults stay until you delete them.
Vaults persist until you delete them.
SOC 2 is in progress.
SOC 2 Type II targeted by January 15, 2027.
Cloud API
The most convenient AI cloud API.
Instead of learning everything about every model released, simply select the model class you want and pay rates based on those tiers. We stay current. We measure the models ourselves, keep up with the latest releases, and select the best option for each model class. You do not change a thing in production. New accounts start with $5 free.
| API | Model name | Input / M | Output / M |
|---|---|---|---|
| LiteGemma 4 E2B | lite | $0.11 | $0.55 |
| BalancedGemma 4 26B | balanced | $0.28 | $1.12 |
| FrontierInkling Small | frontier | $1.05 | $5.25 |
| Embeddings | midium | $0.012 | $0.036 |
| System OneDecider 4B | midium | $0.032 | $0.00 |
When volume is real
Nervous about scaling usage costs?
Start on tokens like anywhere else to build as cheaply as possible. Once you move into production and costs scale with usage, switch to our dedicated capacity to keep costs flat while usage scales.
Cost vs usage
Usage-based pricing
Dedicated capacity
Usage-based pricing climbs. Dedicated capacity stays flat.
Create an account and start using the API today.
The desktop app is there when you want it on a Mac.