Total cost of ownership for an AI gateway
Gateway pricing is usually quoted per seat, per request or per token, and the quote is rarely the largest number in the decision. Four lines decide whether the investment pays back.
Line 1: token spend, and the savings that are actually safe
Caching and compression are the visible savings, and they only count if they cannot bypass inspection.
- Cache hits avoid an upstream call entirely, but blocked or policy-violating content must never be cached
- Semantic compression reduces prompt size only when a gate confirms the meaning survived
- Identity-scoped cache keys prevent one caller's answer being served to another
Line 2: inspection overhead
Inspection costs compute and latency. The variables to model are prompt size, how much of it gets scored, and where scoring runs.
- Local text rules and masking: negligible, microseconds
- Semantic scoring: a model inference, bounded by a per-request gate budget on large prompts
- Attention cost: on very large prompts, compression reduces the tokens the model processes, which usually outweighs inspection overhead
Line 3: operational cost
Every added service is an added on-call surface. A single container in front of your existing monitoring is cheaper to run than a chain of components with independent upgrade cycles.
- One deployment artefact, one configuration surface, one upgrade path
- Fewer credentials to rotate: provider keys centralised rather than spread across agents
- Policy changes applied from an admin interface instead of code deployments
Line 4: the cost of a policy failure
This is the line that dominates, and it is the one that never appears in a feature comparison.
- A leaked credential or a prompt-borne exfiltration event: incident response, notification, remediation
- An audit finding for ungoverned AI usage: repeated evidence gathering for every review cycle
- Shadow AI discovered during an incident rather than by policy: unknown blast radius
A simple model to run
Put the four lines side by side for your own traffic and the decision usually becomes obvious.
- Monthly model spend before and after a realistic cache hit-rate and compression ratio
- Inspection latency added per request, compared with the latency you already accept from the model
- Engineering hours per month to run the platform, at your loaded rate
- A single estimate for one incident, however rough, on the other side of the ledger
Common questions
Yes for measurement. The free edition includes caching, compression, guardrails and the dashboard, and its token allowance is published, so you can measure real savings before committing to a commercial agreement.
They must not. A response that failed inspection, or that carries flagged content, should never enter the cache, and cache keys should be scoped by identity so answers cannot leak across callers.
Model it per rule class: text rules and masking are negligible, semantic scoring depends on prompt size and the gate budget. The published quick start lets you measure both against your own prompts.
Validate it yourself with our Technical Plan
A step-by-step guide to running your own proof of concept and a full head-to-head comparison, in your own environment, so you can decide on evidence rather than on a feature table.