Skip to main content

Data services hardening

French version: ../data-services-hardening.md.

Connection values that carry a secret stay in OpenBao (kv/eai/<env>/control-plane) and are delivered by External Secrets; values-prod.yaml records their required format (controlPlane.secretFormats), asserted by test_platform_security_manifests.py. Before every bao kv put, the operator checks that format.

PostgreSQL (CloudNativePG)​

deploy/data/postgres/cluster.yaml:

  • enableSuperuserAccess: false: no superuser secret is generated; the app owner eai is a plain role, without SUPERUSER or REPLICATION;
  • password_encryption: scram-sha-256, ssl_min_protocol_version: TLSv1.3, connection logging;
  • pg_hba evaluated before the operator defaults: reject every non TLS connection, reject postgres over the network, hostssl with scram-sha-256 for eai on its databases from the pod network (10.42.0.0/16, adapt it), then reject everything else. The operator rules for streaming_replica (certificate) and the local socket come first;
  • client side: EAI_DATABASE_URL ends with ?sslmode=verify-full&sslrootcert=/run/secrets/postgres-ca/ca.crt, the CA being the eai-postgres-ca secret issued by CloudNativePG.

Redis​

deploy/data/redis/values.yaml: plaintext port closed (port 0), tls-port 6379, TLS replication, client certificates required, default user disabled, one ACL user per service (eai-control-plane limited to the eai:rl:* and eai:cache:* keys, apisix limited to the limit-count counters, litellm limited to the litellm:* keys, livekit limited to the keys of LiveKit 1.9.0, which has no prefix, Opik, whose keys have no common prefix, does not use this Redis: it has its own instance redis-opik, deploy/data/redis-opik/values.yaml, ACL user opik, TLS), dangerous commands renamed. Production EAI_REDIS_URL: rediss://eai-control-plane:<password>@redis-master.eai-data.svc.cluster.local:6379/2.

NATS​

deploy/data/nats/values.yaml: TLS on clients and cluster routes, one user per service (control-plane, worker, multimodal) with minimal subject permissions, bcrypt passwords from the nats-users secret. Production EAI_NATS_URL: tls://control-plane:<password>@nats.eai-data.svc.cluster.local:4222.

OpenBao​

The policies in deploy/openbao/policies/ give each component its own paths only: the control plane reads kv/data/eai/+/control-plane, manages kv/data/tenants/* and uses the tenant-* and eai-audit transit keys without export or deletion; LiteLLM reads its provider keys; the portal and workers read their own path. No application policy grants sys/*, transit/export/* or auth/*; only security-officer.hcl (humans, emergency revocation) opens sys/leases/revoke-prefix/*. Everything else is denied by default. Network: deploy/cilium/data-services.yaml opens 8200 only to the enterprise-ai, external-secrets and spire namespaces.

Qdrant​

deploy/data/qdrant/values.yaml: read write and read only API keys from the qdrant-api-key secret (EAI_QDRANT_API_KEY for the control plane), TLS on REST, gRPC and peer to peer traffic, telemetry disabled. EAI_QDRANT_URL switches to https:// in production.

ClickHouse​

deploy/data/clickhouse/users.yaml and config.yaml: default user removed, opik writes to its database opik, eai-writer limited to INSERT on eai_finops.usage_events and eai_telemetry.spans (password CLICKHOUSE_EAI_WRITER_PASSWORD_SHA256, no SELECT, ALTER, CREATE or DROP), eai-reader is read only on eai_finops.*, eai_telemetry.* and opik.* (readonly: 1, quotas and max execution time) for the dashboards, access_management: 0, networks limited to the pods, plaintext ports closed (HTTPS 8443 and native TLS 9440). The control plane exporter (job analytics_export, see analytics-export.md) uses EAI_CLICKHOUSE_USER=eai-writer and EAI_CLICKHOUSE_URL=https://...:8443; the dashboards use eai-reader.

Network policies​

deploy/cilium/data-services.yaml: full deny in eai-data, openbao and eai-security, then one policy per service (PostgreSQL, Redis, NATS, Qdrant, OpenBao, security services) that only accepts the calling components, selected by app.kubernetes.io/component.