Providers, Routing & § 203.
How the gateway selects providers, handles automatic failover, restricts models per key or tag and enforces § 203 StGB.
In your request you specify the model ID rather than the provider: the gateway filters eligible providers for the endpoint family, API format (Anthropic-format routes via /v1/messages, OpenAI-compatible routes via /v1/chat/completions and sibling endpoints), organisation, key and request tags, and returns the answering provider slug in the X-Noirdoc-Provider response header.
from openai import OpenAI
client = OpenAI(base_url="https://api.noirdoc.de/v1")
raw = client.chat.completions.with_raw_response.create(
model="qwen3.8-27b",
messages=[{"role": "user", "content": "Check clause 6 deadlines."}],
)
print(raw.headers.get("x-noirdoc-provider")) # e.g. "aki-io"import OpenAI from "openai";
const client = new OpenAI({ baseURL: "https://api.noirdoc.de/v1" });
const { data, response } = await client.chat.completions
.create({
model: "qwen3.8-27b",
messages: [{ role: "user", content: "Check clause 6 deadlines." }],
})
.withResponse();
console.log(response.headers.get("x-noirdoc-provider"));curl -i https://api.noirdoc.de/v1/chat/completions \
-H "Authorization: Bearer $NOIRDOC_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen3.8-27b",
"messages": [{"role": "user", "content": "Check clause 6 deadlines."}]
}'HTTP/2 200
content-type: application/json
x-noirdoc-provider: aki-ioError responses involving an upstream provider also include X-Noirdoc-Provider; if a streaming provider responds after more than 15 seconds, the gateway emits an SSE comment : provider=<slug> instead (see Streaming).
Provider order, capacity and failover.
For every request, the gateway deterministically orders eligible providers and manages per-provider capacity and pre-stream failover:
| Mechanism | Gateway rule |
|---|---|
Primary sort order (1–5) | 1. Own provider keys (BYOK under Models → Providers, owned_by: "personal") precede managed providers ("platform"). 2. Complete model pricing precedes unpriced routes. 3. Lowest weighted price per 1M tokens (3 × input + 1 × output, excluding cache prices). 4. Higher gateway priority (favouring German DE and sovereign EU data centres at equal price). 5. Longer tenure in the catalogue. |
| Capacity limits | Per-provider concurrency and rate windows prevent overload. When the primary provider is at capacity, the request moves immediately to the next free provider in failover order. If all eligible providers are saturated before forwarding, the gateway waits up to 10 s and then returns 503 capacity_exhausted with Retry-After. |
Automatic failover (max 3 providers) | Switches before any response bytes are sent on connection/timeout failures or upstream HTTP 401, 402, 429 and 5xx – ordering remaining providers by priority first, then price. Once streaming has started, the gateway never switches mid-stream. Noirdoc Shield runs once before attempt 1 so every retry receives the same sanitised payload. |
429 cooldowns & boundaries | On a managed provider 429, the gateway skips that provider for Retry-After clamped to 1–60 s (10 s without a header, 60 s on quota exhaustion; BYOK providers are never paused). Failover never crosses from BYOK to managed providers or from priced to unpriced routes. |
| Final error mapping | Other upstream 4xx responses (plus 401, 402, 403 and 429 on BYOK routes) return immediately without failover as provider_rejected_request. If the final managed provider fails, the gateway returns 429 resource_exhausted (from 429/402), provider_overloaded (503 on OpenAI routes or 529 on /v1/messages and /v1/skills, both with Retry-After), 502 provider_error, 502 provider_unreachable or 504 provider_timeout. |
Model and provider restrictions.
By default, a new API key can call every model enabled for your organisation. Configure restrictions per key under Models → API keys and per request tag under Models → Tags or /v1/tags; GET /v1/models returns the exact set of models permitted for the calling key, organisation and attached tags:
from openai import OpenAI
client = OpenAI(base_url="https://api.noirdoc.de/v1")
models = client.models.list(
extra_headers={"X-Noirdoc-Tags": "mandat-2026-042"}
)
print([m.id for m in models.data])import OpenAI from "openai";
const client = new OpenAI({ baseURL: "https://api.noirdoc.de/v1" });
const page = await client.models.list({
headers: { "X-Noirdoc-Tags": "mandat-2026-042" },
});
console.log(page.data.map((m) => m.id));curl https://api.noirdoc.de/v1/models \
-H "Authorization: Bearer $NOIRDOC_API_KEY" \
-H "X-Noirdoc-Tags: mandat-2026-042" \
| jq -r '.data[].id'When multiple tags are sent in X-Noirdoc-Tags, the gateway takes the intersection: both the model and the provider must be permitted for the organisation, the key and every named tag.
| Setting | Effect & rejection code |
|---|---|
| Allowed models | All models (default, includes future models) or Selected models only. Unlisted model IDs fail before forwarding with 403 model_not_allowed_for_key or 403 model_not_allowed_for_tag. |
| Data residency | Restricts to providers in selected locations (for example DE for German data centres; EU also includes locations in EU member states). Without a selection, every location is permitted. |
| Only providers without CLOUD Act exposure | Excludes providers subject to the US CLOUD Act (403 provider_not_allowed_for_key / provider_not_allowed_for_tag). |
| Only §203-covered providers | Allows only providers bound under § 203 StGB – including your own BYOK providers. |
| Providers | Restricts to selected providers; choosing a single provider pins the key or tag to that route. |
| Starter credit | Before the first confirmed top-up, managed routes allow only models with starter_allowed: true (see Keys, Tags & Budgets). Non-starter providers are skipped, returning 403 starter_model_not_allowed free of charge only if no eligible provider remains; BYOK routes are unaffected. |
Professionals bound by secrecy (§ 203 StGB).
Lawyers, tax advisors, doctors and other professionals bound by secrecy may share confidential data only with service providers bound under § 203 German Criminal Code (StGB). Whether your organisation operates in professional-secrecy mode is set during onboarding on Getting started (active by default until answered, requiring explicit confirmation to turn off, and shown as the §203 client badge under Models → Shield):
| Scope / catalogue value | Managed providers | Own providers (BYOK) | Technical effect |
|---|---|---|---|
Catalogue status covered (§203 covered) | Permitted | Permitted (if marked) | Provider is listed in Annex 4 of the Data Processing Agreement (DPA) and bound under § 203 StGB (contractual details in AI for professionals bound by secrecy). |
Catalogue status on_request (§203 on request) / none | Excluded | Default none | on_request is display-only and treated identically to none by the gateway. |
| Organisation mode | Only covered | All allowed | Filters GET /v1/models, routing, failover and model-less endpoints such as /v1/files. If no eligible provider remains, the gateway returns 403 provider_not_allowed_for_tenant (the wire identifier refers to the organisation). |
Key / tag (Only §203-covered providers) | Only covered | Only covered | Stricter than organisation mode: because the portal provides no covered toggle on BYOK entries, this setting excludes BYOK routes as well (403 provider_not_allowed_for_key or provider_not_allowed_for_tag). |
Provider-side storage: The § 203 check filters providers; it does not disable stateful storage. Whether a managed provider may retain Responses API inputs and outputs for 30 days is controlled separately by Allow Responses API storage at provider (
allow_stored_responsesunder Models → Shield → Advanced Settings, off by default for new organisations; see Chat & Responses).
| Status | Code | Meaning |
|---|---|---|
| 400 | wrong_endpoint_for_model | The model belongs to a different endpoint family or API format than the endpoint called. |
| 403 | model_not_allowed_for_key | The model is not one of this key's allowed models. |
| 403 | provider_not_allowed_for_tenant | The organisation counts as bound by professional secrecy (§ 203 StGB); this excludes every provider that offers this model. |
| 403 | provider_not_allowed_for_key | The key's restrictions (providers, data residency, CLOUD Act, § 203) exclude every provider of this model. |
| 403 | starter_model_not_allowed | The organisation is still using only its starter credit, and the model or its managed provider is not included in starter credit. |
| 403 | model_not_allowed_for_tag | The model is not one of the allowed models of a tag named in X-Noirdoc-Tags. |
| 403 | provider_not_allowed_for_tag | The restrictions of a tag named in X-Noirdoc-Tags (providers, data residency, CLOUD Act, § 203) exclude every provider of this model. |
| 404 | model_not_available | The model ID is not available to the organisation. |
| 429 | resource_exhausted | The provider reported capacity problems or exhausted credit (status code 429 or 402), even after failover across up to three providers. The response contains Retry-After with the wait in seconds. |
| 502 | provider_unreachable | The gateway could not connect to the provider, or the connection dropped. |
| 503 | capacity_exhausted | All providers of the model were at their capacity limit or paused after a 429, even after a wait of up to 10 seconds. The response contains Retry-After with the seconds until a provider is expected to be free again, at most 60, or 5 seconds if that is not known. |
| 503 | provider_overloaded | The provider reported overload (status code 503 or 529), even after failover across up to three providers. The status code is 503 on OpenAI endpoints and 529 on /v1/messages and /v1/skills. The response contains Retry-After with the provider's value between 1 and 60 seconds, otherwise 5 seconds. |
| 504 | provider_timeout | The provider did not answer within the gateway's timeout. The response contains Retry-After with 10 seconds. |