Data residency for AI traffic: what stays in your network
A guardrail that calls a hosted classifier has added a data path you may not have counted in your assessment. Residency questions are easier to answer before the contract than after the incident.
The hidden data path
The usual architecture looks like this: your application sends a prompt to a gateway, the gateway forwards a copy to a moderation service, gets a verdict, then sends the prompt to the model provider. Two vendors now hold your content.
- Inspection service sees the raw prompt and often the response
- Verdicts and sometimes samples are retained for model improvement
- Both paths need their own assessment, DPIA entry and subprocessor listing
What on-premise inspection actually means
Inspection can run in the same process as your traffic, using local rules, local embeddings or a local classifier. Nothing about prompt inspection requires a remote service.
- Rule engines and masking run in-process
- Semantic scoring can use a local model, an embedding endpoint you control, or a classifier you host
- If a scoring gate cannot run, the original content is used rather than an unverified rewrite
Keys, certificates and audit records
Residency is also about the artefacts around the traffic: who holds provider keys, where agent credentials are issued, and where records live.
- Provider keys held server-side in the gateway, injected on forward, never on endpoint agents
- Agent certificates issued by your own CA or an integrated one
- Audit records written to your storage, exportable to your SIEM
Questions to put in an RFP
These five answers will tell you more than a feature matrix.
- Which components see prompt or response content, and where do they run?
- What is retained, for how long, and where?
- What leaves my network for inspection, and can that be disabled?
- What is the failure mode when the inspection dependency is unreachable?
- Can I export decisions to my own logging platform without a per-seat integration?
Keeping vendor models without leaking prompts
Choosing to send prompts to an external model is a business decision. Sending them to an uninspected path is not. Keep the external hop explicit and governed: one egress path, one policy, one record.
Common questions
No. The point is that there is exactly one egress path for AI traffic, it is authenticated and inspected, and the record of what crossed it lives in your environment.
It depends on your regime and data categories, but it removes a class of questions. If no third party processes your prompts, they do not appear in your subprocessor list or your transfer assessment.
They can be produced locally, so the scoring step does not require an external call. That also means semantic policy keeps working if an internet dependency fails.
Validate it yourself with our Technical Plan
A step-by-step guide to running your own proof of concept and a full head-to-head comparison, in your own environment, so you can decide on evidence rather than on a feature table.