B03 - Provider Registry

Group	Technology	Status	Service	Roadmap phase	Contracts
B - LLM management and Model Gateway	Developed (PostgreSQL + OpenBao)	to develop	`llm`	2	-

Back to index - Français

Role and responsibilities

Configured providers, endpoints, regions, write-only credentials, status and kill switch.

Position in the architecture and interactions

Read by the Router (B05) and the LiteLLM config generator; written by tenant admins through the portal or POST /v1/providers. GET /v1/providers/catalog lists presets with status configured / not configured / not connected.

Open source component vs developed part

Fully developed (PostgreSQL table + OpenBao for credentials).

Provider id	Provider	Path through LiteLLM	Admin configuration fields
`openai`	OpenAI	native	api_key, organization, base_url
`anthropic`	Anthropic	native	api_key, base_url
`google_ai_studio`	Google AI Studio / Gemini	native (`gemini/`)	api_key
`vertex_ai`	Vertex AI	native (`vertex_ai/`)	project, location, service account JSON (or workload identity)
`azure_openai`	Azure OpenAI	native (`azure/`)	endpoint, deployment name, api_version, api_key or Entra ID
`azure_ai_foundry`	Azure AI Foundry	adapter selected by endpoint type (OpenAI style or model inference style)	endpoint, api_key or Entra ID, api_version
`aws_bedrock`	AWS Bedrock	native (`bedrock/`)	`aws_region_name`, `aws_access_key_id` + `aws_secret_access_key`, or IAM role (`aws_role_arn` / IRSA), or Bedrock API bearer token (`aws_bearer_token`)
`mistral`	Mistral	native	api_key
`cohere`	Cohere	native (chat, embed, rerank)	api_key
`deepseek`	DeepSeek	native	api_key
`groq`	Groq	native	api_key
`together_ai`	Together AI	native	api_key
`fireworks_ai`	Fireworks AI	native	api_key
`openrouter`	OpenRouter	native	api_key
`ollama`	Ollama	native	base_url
`vllm`	vLLM (local, brick B02)	OpenAI compatible (`hosted_vllm/`)	base_url, optional api_key
`qwen`	Qwen (DashScope)	OpenAI compatible	api_key, base_url
`zai`	Z.AI (GLM)	OpenAI compatible	api_key, base_url
`minimax`	MiniMax	OpenAI compatible	api_key, base_url
`lm_studio`	LM Studio	OpenAI compatible (`lm_studio/`)	base_url
`openai_codex`	OpenAI Codex	OpenAI compatible, OAuth account	OAuth token (brokered)
`github_copilot`	GitHub Copilot	native (`github_copilot/`)	OAuth device token (brokered)
`huggingface`	Hugging Face	native (`huggingface/`)	api_key, endpoint (Inference Endpoints)
`custom`	Custom OpenAI compatible	`openai/` with custom base_url	base_url, api_key, headers
`custom_anthropic`	Custom Anthropic compatible	`anthropic/` with custom base_url	base_url, api_key

AWS Bedrock admin fields: aws_region_name, then one of aws_access_key_id + aws_secret_access_key, aws_role_arn (IRSA / assume role, recommended), aws_web_identity_token + aws_role_arn or aws_bearer_token (Bedrock API key).

Vertex AI admin fields: project, location, service_account_json (or workload identity federation).

Data model

eai.llm_providers(id, tenant_id, name, provider_type, api_base, config, secret_fields, credential_ref, regions, retention, enabled, connected, last_test, created_by, created_at, updated_at); credential_ref points to OpenBao (provider secrets are held in eai.llm_provider_secrets), the value is never stored in clear.

Tables (schema eai, every tenant table carries tenant_id uuid NOT NULL with an RLS policy on current_setting('app.tenant_id', true)::uuid): eai.llm_providers, eai.audit_events.

API endpoints

GET /v1/providers
POST /v1/providers
GET /v1/providers/catalog
GET /v1/providers/{id}
PATCH /v1/providers/{id}
DELETE /v1/providers/{id}
POST /v1/providers/{id}/test
POST /v1/providers/{id}/disable
POST /v1/providers/{id}/enable

POST /v1/providers HTTP/1.1
Host: <control-plane-host>
X-API-Key: eai_xxx_secret
Accept-Language: en
Content-Type: application/json

{"provider_type": "aws_bedrock", "name": "bedrock-eu-west-3", "regions": ["eu-west-3"], "credentials": {"aws_region_name": "eu-west-3", "aws_role_arn": "arn:aws:iam::123456789012:role/eai-bedrock"}}

{"id": "4d2c2cff-7f37-4110-ae93-f8fc8dd771a7", "name": "bedrock-eu-west-3", "provider_type": "aws_bedrock", "scope": "tenant", "api_base": "https://bedrock-runtime.{aws_region_name}.amazonaws.com", "base_url": "https://bedrock-runtime.{aws_region_name}.amazonaws.com", "type": "aws_bedrock", "config": {}, "credentials_set": ["aws_region_name", "aws_role_arn"], "credential_ref": "openbao://llm/providers/4d2c2cff-7f37-4110-ae93-f8fc8dd771a7", "regions": ["eu-west-3"], "retention": {}, "enabled": true, "connected": null, "last_test": null, "status": "configured", "created_at": "2026-10-30T02:38:20.544170+01:00", "updated_at": "2026-10-30T02:38:20.551425+01:00"}

Errors use the common envelope {"error": {"code", "message", "message_i18n", "details", "request_id"}}; lists are paginated with limit and cursor.

SDK usage

Python (enterprise_ai):

from enterprise_ai import EnterpriseAI

client = EnterpriseAI(base_url="<control-plane-url>", api_key="eai_xxx_secret")
items = client.providers.list()

TypeScript (@enterprise-ai/sdk):

import { EnterpriseAI } from "@enterprise-ai/sdk";

const eai = new EnterpriseAI({ baseUrl: "<control-plane-url>", apiKey: process.env.EAI_API_KEY });
const items = await eai.providers.list();

Java (enterprise-ai-java):

EnterpriseAIClient client = EnterpriseAIClient.builder()
    .baseUrl("<control-plane-url>")
    .apiKey(System.getenv("EAI_API_KEY"))
    .build();
var items = client.providers().list();

Go (enterprise-ai-go):

client, _ := eai.NewClient(eai.WithBaseURL("<control-plane-url>"), eai.WithAPIKey(os.Getenv("EAI_API_KEY")))
items, err := client.Providers.List(ctx)

Configuration

Environment: EAI_DATABASE_URL, EAI_OTEL_EXPORTER_OTLP_ENDPOINT.
Helm values: controlPlane.enabled, controlPlane.replicas, controlPlane.resources, controlPlane.config, litellm.enabled, litellm.config.

Security controls

Authentication by OIDC JWT, tenant API key with scopes, or SPIFFE mTLS; tenant resolved from the credential, never from the body.
Authorization through OpenFGA and OPA decisions; every mutation and decision writes a hash-chained eai.audit_events row.
Secrets never returned by the API; TLS everywhere; Cilium network policies restrict who can reach the service.

Control	Enforcement
Provider / model / region allowlists	Per tenant and project in `eai.policies`; the Router only selects allowed triples (provider, model, region)
Data classification	Request `eai.data_classification` must be <= model `data_classification_max`; very sensitive (restricted, secret) data stays on local models (vLLM)
DLP	DLP Gateway (E02) inspects and masks or tokenizes prompts before any remote call
Secret isolation	Credentials write-only, stored in OpenBao, injected into LiteLLM at sync, never returned by the API or logged
TLS	TLS 1.2+ to every provider, mTLS SPIFFE inside the cluster
Provider retention	Registry records the provider retention / zero data retention terms; models without acceptable terms are restricted
Fallback policy	Fallback chains are filtered with the same allowlists: never a forbidden region, model or classification
Cost guardrails	Estimate and quota reservation before execution (`POST /v1/gateway/estimate`, `POST /v1/quotas/reserve`), streaming cut-off during execution when the budget is reached
Logging without exposure	Prompts and completions logged only according to policy, redacted, never with credentials
Kill switch	`POST /v1/providers/{id}/disable` and `POST /v1/models/{model_id}/disable` take effect at the next call

Remote models are allowed for some uses (public or internal data, approved purposes) while very sensitive data stays on local models.

Observability

Metrics: eai_provider_registry_requests_total, eai_provider_registry_errors_total, eai_provider_registry_latency_seconds (Prometheus).
Traces: OpenTelemetry spans eai.providers.* with W3C traceparent propagation, AI spans forwarded to Opik.
Audit actions: provider.create, provider.update, provider.delete, provider.test, policy.evaluate.

FinOps aspects

Operating cost is allocated per tenant and project through labels on Usage Events and Kubernetes resource requests; no direct per-call cost.

Failure modes and resilience

Stateless replicas behind the API Gateway, PostgreSQL HA for state; readiness probe GET /readyz removes unhealthy pods.
External system unavailable: fail closed for security decisions, explicit error code, no silent bypass.

Standalone adoption (client of a client)

A client of a client can adopt this brick alone with the Python SDK sub-client client.providers (TS eai.providers, Java client.providers(), Go client.Providers). Minimal deployment: Control Plane API + PostgreSQL + Keycloak (or API keys), the Control Plane loads every domain of services/control-plane/src/eai/domains; the optional chart components are switched off in deploy/helm/enterprise-ai/values.yaml (vllm.enabled, qdrant.enabled, nats.enabled, workers.enabled).

Acceptance tests

A tenant cannot read or modify another tenant's resources (RLS test).
Every mutation produces an audit event verifiable by GET /v1/audit/verify.
Errors are returned in English and French according to Accept-Language.

Role and responsibilities​

Position in the architecture and interactions​

Open source component vs developed part​

Data model​

API endpoints​

SDK usage​

Configuration​

Security controls​

Observability​

FinOps aspects​

Failure modes and resilience​

Standalone adoption (client of a client)​

Acceptance tests​