OpenAI-compatible API

The /v1 endpoints, chat completions, request headers, error codes, streaming, and cache headers.

The gateway exposes OpenAI-compatible endpoints at its /v1 base URL. Any client that speaks the OpenAI API can point at the gateway instead of the provider.

Endpoints#

Method & pathPurpose
POST /v1/chat/completionsOpenAI-style chat completions (native, and via facade to Anthropic backends)
POST /v1/responsesResponses API relay for newer SDKs; adapted to chat, run through the same pipeline
POST /v1/embeddingsOpenAI-style embeddings (routed to a configured embeddings provider)
GET /v1/modelsList registered models

Common OpenAI-compatible route aliases are accepted alongside the canonical paths, so existing SDKs and tooling connect without changes.

Endpoint Gateway (images, audio, files, fine-tuning, batches)#

Beyond chat, messages, and embeddings, the gateway can expose the remaining OpenAI-compatible surfaces. These are opt-in and disabled by default; disabled endpoints return 404 with code endpoint_disabled. Enable each kind in Settings -> Endpoint Gateway:

SurfaceGateBehavior
/v1/images/generations, edits, variationsimagesJSON prompt scanned, masked, and compressed like a chat prompt; response metadata scanned; inspection-gated cache for generations
/v1/audio/speechaudioInput scanned like a chat prompt, then relayed
/v1/audio/transcriptions, translationspassthroughMultipart governed pass-through with key injection and audit
/v1/filesfilesText files decoded and scanned; violations block the upload, binary files pass through
/v1/fine_tuning/jobs, /v1/batches (+ cancel)passthroughGoverned pass-through with key injection and audit

The media content itself (pixels, audio waveforms) is never analyzed, only the surrounding text. Every request records the endpoint kind in the transaction log.

Request headers#

HeaderPurpose
Authorization: Bearer <key>Client authentication and/or upstream key per the configured auth and key modes
X-Agent-IDAgent identity, tracking, routing rules, risk, cache scoping
X-User-IdUser identity for tracking (authenticated token sub claims are preferred)
x-aifw-cache-refreshBypass the cache lookup for this request (the write is still allowed)

Standard request#

POST /v1/chat/completions
{
  "model": "gpt-4o",
  "messages": [
    { "role": "system", "content": "You are a helpful assistant." },
    { "role": "user", "content": "Hello!" }
  ],
  "stream": false
}
FieldNotes
modelOptional, omitted values are filled from the configured default model
messagesStandard chat messages; tool_calls/tool sequences pass through byte-identical
streamtrue applies the effective streaming mode (buffered or live)

Error codes#

Blocked or rejected requests return HTTP 400 with a structured error:

{
  "error": {
    "message": "The transaction was intercepted and blocked by corporate AI security policy.",
    "type": "guardrail_violation",
    "code": "prompt_injection_detected",
    "details": { "policy_id": "BUILTIN-JAILBREAK", "timestamp": "2025-01-15T10:00:00Z" }
  }
}

Common codes:

CodeMeaning
guardrail_violationBlocked by a rule (built-in or custom); details carry the policy ID
prompt_injection_detectedJailbreak/prompt-injection rule matched
semantic_rule_violation:<rule>Semantic tier scored above the block threshold
model_disabledThe model is registered but not enabled
model_not_registeredStrict mode is on and the model isn't in the registry
backend_not_configuredNo backend URL anywhere for the resolved model
default_policy_blockDeny-by-default fired (nothing matched, default action is block)

Blocked responses also carry the X-Gateway-Security: Violated header. Auth failures return 401; unknown routes 404.

Streaming#

With stream: true, responses use server-sent events exactly like the OpenAI format, ending with data: [DONE]. The gateway applies the effective streaming mode (see identity & access):

  • buffered (default): the full response is scanned before any token is delivered; a violation returns 400 before delivery.
  • stream: tokens flow live; a mid-stream violation terminates the SSE stream with a guardrail_violation error event. Tokens already delivered cannot be retracted.

Caching#

When a cache scope is configured (see the reliability guide), responses may be served from the completion cache. Cache hits are recorded in the transaction log, and cached bodies still pass through response inspection on replay.

Related