Enterprise AI Platform - Architecture overview

Français - Diagram (HTML + SVG) - Index

1. Layers

Layer	Content	Bricks
Control Center	Next.js portal, FastAPI Control Plane API, CLI enterprise-ai, SDKs (Python, TypeScript, Java, Go, Realtime), APIs, GitOps. Administration, Developer Portal, Security, Governance, FinOps, Audit	A01, A02, A10 to A14
Identity, Access and Policy Enforcement	Keycloak, SPIFFE/SPIRE, OpenFGA, OPA, Policy Enforcement Points, OpenBao, cert-manager	A03 to A09
AI Model Gateway	LiteLLM (unified gateway), vLLM (local inference engine), remote providers, registries, router, resolver, effort and context, token accounting	B01 to B14
Agent Runtime	Navin architecture: AgentLoop, AgentRunner, SubagentManager, RunPlan, MessageBus; server worker python -m eai.worker; sandbox, kill switch, replay; optional LangGraph and Temporal adapters	C01 to C15
Capability Gateway	Tools, MCP, Skills, Plugins	D01 to D08
Integration Gateway	Connectors, Credential Broker, Webhooks, Channels, Events, API Gateway	D09 to D14
Data and Privacy Gateway	Classification, DLP, Tokenization, Encryption	E01 to E04
Knowledge Gateway	Document processing, RAG, Qdrant, Context, Memory, Lineage, Retention	E05 to E13
Multimodal Gateway	Vision, Image, Video, STT, Diarization, TTS, Voice Live, VAD	F01 to F12
Quality and Evaluation	Ragas, Promptfoo, garak, Quality Orchestrator	G09, G10, G13, B12
Unified Observability, Audit and FinOps	OpenTelemetry, OTel Collector, Opik, ClickHouse, Prometheus, OpenSearch, Alertmanager, WORM audit, quotas, ledger, pricing	G01 to G08, G11, G12, G14, H01 to H04
Sovereign Infrastructure	Kubernetes, Cilium, KEDA, Argo CD, OpenTofu, Kyverno, Trivy, Syft, Cosign, Velero, CI/CD, Harbor, PostgreSQL, Redis, Qdrant, Object Storage, GPU	H05 to H16, E07, E08, E13

Arrows in the diagram are logical interactions, not a single HTTP pipeline. vLLM is a local inference engine; LiteLLM is the unified gateway to local and remote models: vLLM is one upstream of LiteLLM.

2. Non negotiable principles

1. One Control Plane: a single source of truth for registries, policies, quotas, prices and audit (/v1, see API).
2. Several specialized runtimes: agents (Navin architecture of the Python SDK, optional LangGraph and Temporal adapters), realtime voice (LiveKit), batch and media (multimodal workers over NATS), periodic jobs (python -m eai.worker), inference (vLLM). No runtime owns governance data.
3. Mandatory gateways: models, tools, MCP, knowledge, data and multimodal are reached only through their gateway. Agents never hold provider keys and never call LiteLLM, vLLM or a connector directly.
4. Unified telemetry: every call carries W3C traceparent, produces OpenTelemetry spans, a Usage Event and a hash-chained audit event.
5. SDKs impose development contracts (manifests, typed calls, client-side validation of contracts/v1), but gateways, network policies (Cilium) and identities (SPIFFE, Keycloak) really enforce security. An SDK bypass gains nothing.
6. Phase 11 requirements (HA, DR, load, security, compliance) apply from phase 1 (roadmap (catalog/roadmap.yaml)).

3. Multi-tenancy model

platform (operator, platform_admin)
  -> tenant (large account client): own IdP realm or federation, quotas, policies, providers, keys
       -> projects (X-Project)
            -> environments dev | test | staging | prod (X-Environment)

• The tenant is resolved from the credential, never from the body. PostgreSQL RLS on tenant_id isolates every table (eai.tenants, eai.projects, eai.environments, eai.llm_models, eai.llm_providers, eai.policies, eai.audit_events...).
• OpenFGA tuples model project and resource relations; OPA policies are layered platform -> tenant -> project.
• Clients of clients consume either the whole platform (portal + all gateways) or a single brick with its SDK module (for example only enterprise_ai.llm.gateway for the Model Gateway, or only enterprise_ai.dlp), with a minimal deployment: Control Plane + PostgreSQL + the brick component. Example sub-client: client.gateway of the Python SDK.

4. Deployment topologies

Topology	Description
Single brick	Control Plane + PostgreSQL + one component (e.g. LiteLLM gateway), API keys auth
Shared SaaS	One cluster, many tenants isolated by RLS, OpenFGA, Cilium namespaces
Dedicated tenant	Dedicated namespace or cluster per large account, own OpenBao mount and GPU pool
Sovereign on premises	Air-gapped cluster, only local models (vLLM), Harbor mirror, Argo CD from internal Git
Hybrid	Local vLLM for restricted data, remote providers allowed by policy for other classifications

All are delivered by Argo CD from Helm charts, signed images (Cosign), Kyverno admission and Velero backups.

5. Request flows

5.1 Model gateway call

1. Client or agent: POST /v1/gateway/chat/completions with the eai extension.
2. Authentication (JWT, API key or SPIFFE) and tenant resolution.
3. Policy Enforcement (A07 -> OPA/OpenFGA), decision recorded.
4. DLP Gateway (E02) on the prompt: detect, mask, tokenize or block.
5. Capability Resolver and Effort Policy (B06, B07): native parameters or explicit rejection.
6. Context Window Manager (B08): B_context = W_model - B_output - B_reasoning - B_safety.
7. Estimate and quota reservation (H03, H01).
8. Router (B05) selects an allowed (provider, model, region); LiteLLM (B01) calls vLLM or the remote provider; failover (B14) stays inside allowlists.
9. Token Accounting (B09) -> Usage Event -> measured cost, quota commit, audit event, spans to Opik (see ARCHITECTURE.md).

5.2 Agent run

POST /v1/agents/{id}/runs creates a pending run -> the client AgentLoop.serve() process picks it up -> AgentRunner runs the turn: model call through the Model Gateway, then tool calls through the Tool Gateway (policy, DLP, approval), until the final answer -> checkpoints and heartbeats at every step -> human-in-the-loop via approvals -> kill switch available at any time (the heartbeat answers action=stop) -> Execution Contract (GET /v1/runs/{id}).

Runtime architecture (Navin, no declared graph):

Element	Role
enterprise_ai.loop.AgentLoop	Message intake (bus, pending runs, watched sessions), context, sessions, crash resume (resume), stop (shutdown), cron and heartbeat, serve()
enterprise_ai.runtime.AgentRunner	One turn: model call then tool calls until the final answer, with the continuation rules (work remaining, user already said go, verify before finishing)
enterprise_ai.subagents.SubagentManager	Parallel sub-agents, monitoring and injection of their results into the parent turn
enterprise_ai.loop.plan.RunPlan	Plan with dependent steps, automatic advance to the next ready step
enterprise_ai.runtime.bus.MessageBus	Message bus between channels and engine (in memory, or NATS with NatsMessageBus)
python -m eai.worker	Server periodic jobs: scheduler tick, event outbox drain, quota reservation expiry, stale heartbeats, retention

LangGraph (enterprise_ai.runtime.langgraph) and Temporal (enterprise_ai.runtime.temporal) are optional adapters that reuse the same steps.

5.3 RAG query

POST /v1/knowledge-bases/{id}/query -> ACL filter (OpenFGA) -> Qdrant retrieval -> classification check -> Context Manager plan -> model call through the gateway -> answer with sources; lineage event.

5.4 Voice live session

POST /v1/realtime/sessions returns a LiveKit token, ws url and session id. Audio flows over WebRTC (not HTTP), VAD (Silero) segments it, STT (faster-whisper) -> agent or model -> TTS (Piper); events on /v1/realtime/ws/{session_id} (transcript.delta, response.audio.delta, usage.update). Usage is metered per second and per token.

5.5 FinOps lifecycle

Stage	Source	API
Estimated	price card x planned units, before the call	POST /v1/gateway/estimate, POST /v1/finops/estimate
Measured	Usage Event after the call (tokens, seconds, GPU time)	POST /v1/finops/usage
Billed	provider invoice or export import, reconciliation	POST /v1/finops/reconciliation/run, GET /v1/finops/reconciliation/report

6. Contracts overview

Shared JSON Schemas in contracts/v1/ are the source of truth for the Control Plane, gateways, portal, SDKs and CLI: common, agent-manifest, model-manifest, tool-manifest, skill-manifest, plugin-manifest, connector-manifest, channel-manifest, policy-contract, execution-contract, context-contract, memory-contract, multimodal-contract, evaluation-contract, usage-event, audit-event. They are validated by POST /v1/contracts/{name}/validate, applied by POST /v1/manifests/apply, and checked client-side by every SDK.

Brick catalog: catalog/bricks.yaml (served by GET /v1/catalog/bricks).