Network and DNS hardening
French version: ../network-dns-hardening.md.
Principle
Every namespace starts from a full deny, ingress and egress, DNS included. Each workload then opens only the flows it needs. Chart rules are rendered by deploy/helm/enterprise-ai/templates/control-plane/networkpolicy.yaml (Cilium flavour by default, plain Kubernetes NetworkPolicy with global.networkPolicy=kubernetes). Data and security namespaces are covered by deploy/cilium/data-services.yaml, cluster-wide policies by deploy/cilium/baseline.yaml.
| Policy | Scope | Effect |
|---|---|---|
eai-default-deny | enterprise-ai namespace | ingress: [{}] and egress: [{}]: no flow, not even DNS |
eai-deny-metadata | every pod of the namespace | egressDeny to 169.254.169.254/32, 169.254.0.0/16, fd00:ec2::254/128 |
eai-deny-cloud-metadata | whole cluster (CiliumClusterwideNetworkPolicy) | same deny for every workload in any namespace |
eai-deny-kube-apiserver | every pod except control-plane | egressDeny to the kube-apiserver entity |
default-deny | eai-data, openbao, eai-security | full deny, then one policy per service |
egressDeny rules always win over allow rules: an SSRF bug in an application cannot read instance credentials (AWS IMDS, GCP, Azure, OpenStack).
Chart flow matrix
| Workload | Allowed ingress | Allowed egress |
|---|---|---|
| control plane | APISIX, portal, observability on 8080 | PostgreSQL, NATS, Redis, Qdrant, Temporal, LiveKit (eai-data), external S3 storage 443, OpenFGA, OPA, Keycloak (eai-security), OpenBao 8200, OTLP, Opik, ClickHouse, LiteLLM 4000, vLLM 8000, Ollama 11434, Kubernetes API, provider FQDNs on 443 |
| background worker | none | PostgreSQL, NATS, Redis, OpenBao, OTLP |
| migration | none | PostgreSQL, OpenBao |
| portal | ingress controller, APISIX on 3000 | control plane, Keycloak, OTLP |
| LiteLLM | control plane, observability on 4000 | vLLM, Ollama, PostgreSQL, Redis, OpenBao, OTLP, provider FQDNs on 443 |
| vLLM | LiteLLM, control plane, observability on 8000 | none (weights preloaded, HF_HUB_OFFLINE=1) |
| multimodal workers | none | NATS, external S3 storage 443, OTLP |
Only the control plane and LiteLLM reach the Internet, through an FQDN list (networkSecurity.providerFQDNs in values.yaml, same as deploy/cilium/egress-providers.yaml). Set networkSecurity.providerEgress.controlPlane or .litellm to false to close one side.
DNS: per workload L7 allowlist
The Cilium DNS proxy enforces a matchName / matchPattern list per workload:
- internal names limited to the namespaces actually called, for example
*.eai-data.svc.cluster.local; - external names limited to the provider FQDNs, for the control plane and LiteLLM only;
- vLLM has no DNS access at all.
Any other query gets REFUSED: a compromised pod can neither resolve an arbitrary domain nor exfiltrate data encoded in subdomains (DNS tunneling). Cilium unions the DNS rules of every policy selecting a pod, so no policy may contain matchPattern: "*". The eai-dns-visibility policy in baseline.yaml is now limited to *.svc.cluster.local, and test_platform_security_manifests.py rejects any * pattern.
Application side: use fully qualified names (nats.eai-data.svc.cluster.local), as values-prod.yaml does. With short names the resolver walks the search domains, those intermediate queries are refused and resolution slows down.
CoreDNS and NodeLocal DNSCache
- No open resolver: the
kube-dnsService staysClusterIP, never exposed throughLoadBalancerorNodePort(the Kyverno policyeai-restrict-service-exposureblocks NodePorts outsideingressandapisix). CoreDNS listens on the pod network only. - Restricted upstream forwarding: in the Corefile,
forward . <corporate resolvers>with explicit addresses, never the node/etc/resolv.confwhen it points to a public resolver. Addpolicy sequentialandhealth_check 5s. - DNSSEC or DoT to the resolver: either the upstream resolver validates DNSSEC (Unbound
auto-trust-anchor-file, BINDdnssec-validation auto), or CoreDNS speaks DoT to it:forward . tls://10.0.0.53 { tls_servername <dns-host> }. Do not enable the CoreDNSdnssecplugin (it signs, it does not validate). - No external recursion for pod records: the
cluster.localblock uses thekubernetesplugin withpods verified(notpods insecure), soa-b-c-d.ns.pod.cluster.localonly answers for an existing pod. - Logging:
logplugin filtered ondenialanderror, shipped to OpenSearch (indexeai-logs-*) by the OpenTelemetry Collector; the Cilium DNS proxy also reports refusals in Hubble (hubble observe --verdict DROPPED --protocol dns). - NodeLocal DNSCache: the
eai-dns-visibilitypolicy and the workload rules selectk8s-app: kube-dns. With NodeLocal DNSCache, add thek8s-app: node-local-dnsselector to the same rules (same port 53, same L7 rules); the local cache forwards to CoreDNS over TCP and has no other upstream. - Rate limiting:
cache 30plus theratelimitexternal plugin or the NodeLocal DNSCache query limit protect CoreDNS from a pod flooding DNS.
Offline verification
helm template eai deploy/helm/enterprise-ai -n enterprise-ai -f deploy/helm/enterprise-ai/values-prod.yaml | kubeconform -strict -summary
services/control-plane/.venv/bin/python -m pytest services/control-plane/tests -q -k platform
On a test cluster (never in production without a change window): hubble observe --namespace enterprise-ai --verdict DROPPED must show the expected drops, and a kubectl exec from a portal pod to 169.254.169.254 must fail.