Model choice used to be an engineering preference. In 2026 it is a governance and P&L decision — especially for operators running agents on real volume. This article lays out the cost math we use with clients: open-weight models by default, frontier when they earn their place, with numbers labelled illustrative unless tied to published product copy.
For architecture context, read the open hybrid stack page and our post on model stack as governance.
Two buckets, one routing layer
| Bucket | Examples | Typical workload |
|---|---|---|
| Open-weight core | Llama, Mistral, Qwen, GLM-class | Triage, tagging, extraction, standard drafts, high volume |
| Frontier APIs | Claude, GPT, Gemini | Complex chains, delicate tone, ambiguous multi-step reasoning |
The mistake is picking one vendor for everything. The discipline is per-call routing with logs — what the AI Operating Layer is built to hold.
Illustrative per-call economics
Vendor prices move weekly; treat the table as order-of-magnitude, verify before budgeting:
| Task class | Open routing (illustrative) | Frontier routing (illustrative) | Ratio |
|---|---|---|---|
| Single-pass classification | Low $/1M tokens | Medium $/1M tokens | ~3–5× |
| Long-thread summarisation | Medium | High | ~4–8× |
| Multi-tool agent step chain | Medium–high | High | ~6–12× |
Our product copy uses ~6–12× as a hybrid-stack reference when the same workflow is routed entirely to frontier vs a well-chosen open model for the bulk of calls — not a promise in your invoice.
Blended-cost math (~80/20)
Suppose a workflow with 10,000 model calls/month, normalised cost 1 unit per open call and 8 units per frontier call (illustrative 8×):
| Strategy | Calculation | Relative inference cost |
|---|---|---|
| 100% open | 10,000 × 1 | 10,000 |
| 100% frontier | 10,000 × 8 | 80,000 |
| 80% open / 20% frontier | 8,000×1 + 2,000×8 | 24,000 (~70% below all-frontier) |
Real mixes depend on which steps are frontier-worthy. Finance cares about the blended line, not the demo that used only the premium model.
Data residency and the open option
Frontier APIs often mean data leaves your chosen region under vendor terms. Capable open models can run:
- In your cloud tenant (EU, UK, US as agreed)
- Via EU-hosted inference providers
- On-prem where capital and ops justify it
Residency is not free — but it is choosable in a way single-vendor frontier defaults sometimes are not. That matters for hospitality guest data, HR, and health-adjacent workflows.
When frontier wins (honest list)
Route premium APIs when all or most apply:
- More than three dependent tool steps with failure cost
- Reputational tone risk (complaint, executive B2B comms)
- Ambiguity where misclassification costs more than token delta
- Evaluation shows open model misses agreed accuracy on representative cases
If none apply, open is usually the correct first answer — prove it in pilot evals, not vendor marketing.
When open wins
- High-volume classification and tagging
- Structured extraction with human validation on exceptions
- Standard drafts from approved templates
- Pre-filter before a smaller frontier “review pass” on hard cases
Pattern: open first pass, frontier second pass on flagged items — cuts cost without hiding hard cases.
Operational requirements (unglamorous, mandatory)
Routing without observability is guessing:
- Log model, tokens, estimated cost, tool name per call
- Dashboard or weekly export finance can read
- Review queue metrics (approval rate, correction rate) tied to model changes
- Re-run eval set when prompts, tools, or models change
This is the same bar we set in AI governance engagements — model choice is a policy object, not a developer secret.
Cost on the whiteboard before the pilot
Discovery should produce:
- Workflow diagram with model assignment per step
- Illustrative monthly inference band (open, frontier, blended)
- Criteria to promote a step to frontier or demote to open after review
Project figures from when this note was written (background, not a live offer) sit in the implementation cost breakdown. Current public work is a Web & eCommerce quote or US → EMEA.
Common failures
- All frontier because the demo looked smarter — bill scales linearly with volume.
- All open on tone-critical external drafts — correction labour eats savings.
- No eval set — you discover failure in week three of season, not in lab.
- Routing in prompts only — without gateway logs, finance and security cannot audit.
If model economics block your business case, ask for a package quote or read the open stack page kept as background.
Frequently asked questions
Are open models good enough for production agents?
For classification, extraction, and standard drafts — often yes, when tools and evaluation are well designed. Frontier models earn their place on complex reasoning, delicate tone, or long dependent tool chains — not on every call by default.
What is a realistic open/frontier split?
Illustrative design pattern: ~80% open / ~20% frontier by call volume can land ~70% below all-frontier inference for similar workload — not a guaranteed saving; depends on task mix and routing discipline.
Does self-hosting open models eliminate cost?
It changes the curve: no per-token vendor bill, but GPU/hosting capital and ops. Mid-market teams often start with hosted open APIs in an agreed region, then evaluate self-host if volume justifies.
How do we decide per workflow?
Discovery maps sensitivity, latency, language, and error cost. Put per-workflow model choice and expected monthly inference on the whiteboard before the pilot — same discipline we describe on the open-stack product page.