A14 - Incident Management

Group	Technology	Status	Service	Roadmap phase	Contracts
A - Governance, identity and security	Developed + SIEM	to develop	incidents	10	audit-event

Back to index - Français

Role and responsibilities

AI incidents lifecycle and containment actions (suspend agent/model/provider, rotate key), SIEM forwarding.

SOC and AI-SDR (kit section 11, domain eai.domains.soc): detections from internal and external sources are normalized, correlated by user, agent or tenant into one incident of category soc, scored by the risk engine (severity, confidence, reasons on the timeline), then answered with typed reactions only (/v1/soc/orders) through the Response Policy Engine and its proportionality guard. Each containment is a row of the Kill Switch Registry (eai.kill_switches, desired and observed state) enforced by the policy decision point, the authentication and the agents runtime, and becomes SUSPENDED_VERIFIED only on a receipt. Recovery needs two distinct approvers and a re-authorization; incident evidence is sealed with a chain of custody. Runbook: SOC, Kill Switch and recovery.

Data model

Tables (schema eai, every tenant table carries tenant_id uuid NOT NULL with an RLS policy on current_setting('app.tenant_id', true)::uuid): eai.incidents, eai.incident_actions, eai.audit_events.

API endpoints

• GET /v1/incidents
• POST /v1/incidents
• GET /v1/incidents/{id}
• PATCH /v1/incidents/{id}
• POST /v1/incidents/{id}/actions

POST /v1/incidents HTTP/1.1
Host: <control-plane-host>
X-API-Key: eai_xxx_secret
Accept-Language: en
Content-Type: application/json

{"title": "Invoice agent returns PII in answers", "severity": "high", "category": "data_leak", "description": "Customer emails appear in agent responses"}

{
  "id": "3421a88d-c114-4b2c-9ec0-addc883aefbd",
  "tenant_id": "18de5695-21f9-4053-a4ea-beb4bf2a5932",
  "title": "Invoice agent returns PII in answers",
  "description": "Customer emails appear in agent responses",
  "severity": "sev2",
  "status": "open",
  "category": "data_leak",
  "resources": [],
  "assignee": null,
  "created_by": "alice",
  "created_at": "2026-10-30T01:35:52.825765Z",
  "updated_at": "2026-10-30T01:35:52.825765Z",
  "resolved_at": null,
  "timeline": [
    {"kind": "created", "message": "incident opened with severity sev2", "data": {"severity": "sev2", "resources": []}, "actor": "alice", "at": "2026-10-30T01:35:52.828583Z"}
  ],
  "actions": []
}

Errors use the common envelope {"error": {"code", "message", "message_i18n", "details", "request_id"}}; lists are paginated with limit and cursor.

SDK usage

Python (enterprise_ai):

from enterprise_ai import EnterpriseAI

client = EnterpriseAI(base_url="<control-plane-url>", api_key="eai_xxx_secret")
items = client.incidents.list()

TypeScript (@enterprise-ai/sdk):

import { EnterpriseAI } from "@enterprise-ai/sdk";

const eai = new EnterpriseAI({ baseUrl: "<control-plane-url>", apiKey: process.env.EAI_API_KEY });
const items = await eai.incidents.list();

Java (enterprise-ai-java):

EnterpriseAIClient client = EnterpriseAIClient.builder()
    .baseUrl("<control-plane-url>")
    .apiKey(System.getenv("EAI_API_KEY"))
    .build();
var items = client.incidents().list();

Go (enterprise-ai-go):

client, _ := eai.NewClient(eai.WithBaseURL("<control-plane-url>"), eai.WithAPIKey(os.Getenv("EAI_API_KEY")))
items, err := client.Incidents.List(ctx)

Configuration

• Environment: EAI_DATABASE_URL, EAI_OTEL_EXPORTER_OTLP_ENDPOINT.
• Helm values: controlPlane.enabled, controlPlane.replicas, controlPlane.resources, controlPlane.config.

Security controls

• Authentication by OIDC JWT, tenant API key with scopes, or SPIFFE mTLS; tenant resolved from the credential, never from the body.
• Authorization through OpenFGA and OPA decisions; every mutation and decision writes a hash-chained eai.audit_events row.
• Secrets never returned by the API; TLS everywhere; Cilium network policies restrict who can reach the service.

Observability

• Metrics: eai_incident_management_requests_total, eai_incident_management_errors_total, eai_incident_management_latency_seconds (Prometheus).
• Traces: OpenTelemetry spans eai.incidents.* with W3C traceparent propagation, AI spans forwarded to Opik.
• Audit actions: incident.create, incident.update, policy.evaluate.

FinOps aspects

Operating cost is allocated per tenant and project through labels on Usage Events and Kubernetes resource requests; no direct per-call cost.

Failure modes and resilience

• Stateless replicas behind the API Gateway, PostgreSQL HA for state; readiness probe GET /readyz removes unhealthy pods.
• External system unavailable: fail closed for security decisions, explicit error code, no silent bypass.

Standalone adoption (client of a client)

A client of a client can adopt this brick alone with the Python SDK sub-client client.incidents (TS eai.incidents, Java client.incidents(), Go client.Incidents). Minimal deployment: Control Plane API + PostgreSQL + Keycloak (or API keys), the Control Plane loads every domain of services/control-plane/src/eai/domains; the optional chart components are switched off in deploy/helm/enterprise-ai/values.yaml (vllm.enabled, qdrant.enabled, nats.enabled, workers.enabled).

Acceptance tests

1. A tenant cannot read or modify another tenant's resources (RLS test).
2. Every mutation produces an audit event verifiable by GET /v1/audit/verify.
3. Errors are returned in English and French according to Accept-Language.