Install the Navin Platform on Kubernetes with Helm

This page installs the Navin Platform on a Kubernetes cluster: the cluster operators, the data and security services, then the enterprise-ai chart (Control Plane API, worker, portal, LiteLLM, vLLM, media workers). It is done by a platform engineer with cluster administration rights. Count one to two days for a first production cluster, most of it for the secrets and the certificates.

In production, Argo CD installs everything from the repository in four layers (deploy/argocd/applicationset.yaml): infra, data, platform and apps. A direct helm upgrade --install of the chart is also described, for a cluster whose dependencies already exist.

Before you begin

• Kubernetes 1.27 or newer: the chart declares kubeVersion: ">=1.27.0-0". Check: kubectl version.
• A CSI storage driver with a storage class named fast-ssd (or change the name in deploy/data/*), and local-nvme for the vLLM model cache of the on premises GPU profile. See Sizing.
• An S3 compatible object storage reachable over HTTPS (Ceph RGW or any S3 API) for the WORM audit archive, the media artifacts, the CloudNativePG and Velero backups.
• A Harbor registry (or an equivalent OCI registry) holding the images, mirrored and signed, and a Cosign key pair: the Kyverno policies refuse unsigned platform images.
• A DNS zone for the public names (<control-plane-host>, <portal-host> in the manifests) and a public or company certificate authority (TLS).
• NVIDIA GPU nodes labelled nvidia.com/gpu.present=true when vLLM or the media workers run on GPU.
• Tools on the workstation: kubectl, helm 3.16 or newer, tofu or terraform, argocd, bao (OpenBao CLI).
• The flows of Network ports and flows open.

Components installed by the layers

Layer	Components and pinned versions
bootstrap (Terraform module platform-bootstrap)	Cilium 1.16.5, cert-manager v1.16.2, Kyverno chart 3.3.4, KEDA 2.16.1, CloudNativePG chart 0.23.0, Velero chart 8.1.0, NVIDIA GPU Operator v24.9.1, Argo CD chart 7.7.11, namespaces with their Pod Security level
infra	Cilium policies, cert-manager issuers, ToolHive CRDs 0.59.0, Kyverno policies, KEDA objects, Velero schedules, External Secrets 2.14.0, kube-prometheus-stack 92.16.0, SPIRE CRDs 0.6.1 and SPIRE 0.30.4
data	OpenBao chart 0.30.6, platform secrets, PostgreSQL (CloudNativePG), Redis operator 0.27.0 and Redis replication 0.17.1, NATS 2.15.1, Qdrant 1.20.0, Temporal 1.7.0, ClickHouse operator 0.27.4, Keycloak (keycloakx 7.4.0) and its realm, OpenFGA 0.3.19, OPA
platform	APISIX 2.18.2 and its routes, LiveKit, OpenSearch 3.9.0, Alloy 1.14.4, OpenTelemetry Collector 0.178.1, Jaeger 4.16.1, Opik 2.2.137, alert rules, ToolHive Operator 0.59.0
apps	deploy/helm/enterprise-ai

Chart versions: deploy/terraform/modules/platform-bootstrap/variables.tf (chart_versions) and deploy/argocd/applicationset.yaml.

Steps

1. Install the cluster foundation. The Terraform module platform-bootstrap installs the operators every other layer needs (network, certificates, admission, autoscaling, PostgreSQL operator, backups, GPU, Argo CD), creates every namespace with its Pod Security level and registers the cluster in Argo CD with the label eai.io/environment.

   cd deploy/terraform/envs/onprem-gpu
   tofu init -backend-config=backend.hcl
   tofu plan -var kube_context=<kube-context>
   tofu apply

   <kube-context>: dedicated kubeconfig context of the target cluster (default eai-onprem, kubeconfig ~/.kube/eai-onprem.yaml). backend.hcl: your partial configuration of the s3 backend that main.tf declares. The module also needs cosign_public_key, the PEM public key Kyverno uses to verify the images (-var or a .tfvars file).

   Expected result: kubectl get pods -n kube-system -l k8s-app=cilium, -n cert-manager, -n kyverno, -n keda, -n argocd show running pods, and kubectl get ns lists enterprise-ai, eai-data, eai-security, observability, openbao, eai-sandbox.

   If it fails: tofu plan shows the resource in error. A chart that cannot be downloaded means the cluster has no access to the chart repositories: use the mirror of Air-gapped installation (helm_repository_mirror).

2. Prepare the storage classes. The data services ask for fast-ssd; the vLLM cache of the on premises profile asks for local-nvme.

   kubectl get storageclass

   Expected result: fast-ssd (and local-nvme with GPU) in the list.

   If it fails: create the classes on your CSI driver, or replace storageClass and storageClassName in deploy/data/*/values.yaml, deploy/data/postgres/cluster.yaml and vllm.modelCache.storageClass.

3. Register the GitOps project and the ApplicationSet. The AppProject limits the sources and destinations; the ApplicationSet generates one Application per component for every cluster carrying the environment label.

   kubectl -n argocd apply -f deploy/argocd/appproject.yaml -f deploy/argocd/repository-toolhive.yaml \
     -f deploy/argocd/applicationset.yaml

   Expected result: argocd app list -l eai.io/environment=<env> lists the Applications of the four layers.

   If it fails: an empty list means the cluster secret of Argo CD has no eai.io/environment label; add it with argocd cluster add <kube-context> --label eai.io/environment=<env>.

4. Synchronize the infra layer. In dev and staging the synchronization is automatic; in prod it is manual, layer after layer.

   argocd app sync -l eai.io/environment=prod,eai.io/layer=infra
   argocd app wait -l eai.io/environment=prod,eai.io/layer=infra --health --timeout 1800

   Expected result: every Application of the layer is Synced and Healthy; kubectl get clusterissuer lists eai-internal-ca.

   If it fails: argocd app get <application> shows the resource in error. A missing CRD is retried by Argo CD (10 attempts, up to 10 minutes).

5. Synchronize the data layer, then initialize OpenBao. OpenBao starts sealed at the first installation: the secrets of every other component wait for it.

   argocd app sync -l eai.io/environment=prod,eai.io/layer=data
   kubectl -n openbao exec -ti openbao-0 -- bao operator init
   kubectl -n openbao exec -ti openbao-0 -- bao operator unseal
   bash deploy/openbao/bootstrap.sh

   bao operator unseal is run once per unseal key share requested by init, on each OpenBao pod. bootstrap.sh enables KV v2 (kv), Transit, the Kubernetes auth method and the policies.

   Expected result: bao status shows Sealed false.

   If it fails: keep the unseal keys and the root token out of the cluster, in the safe of the bank. A pod that stays sealed after a restart needs the unseal keys again (or an auto unseal configured in deploy/openbao/values.yaml).

6. Write the secrets. Every third-party service reads its credentials from a Secret made by External Secrets from OpenBao. The Control Plane reads its own from kv/eai/<env>/control-plane.

   bao kv put kv/eai/prod/control-plane \
     EAI_DATABASE_URL='postgresql://eai:<password>@eai-postgres-rw.eai-data.svc:5432/eai?sslmode=verify-full&sslrootcert=/run/secrets/postgres-ca/ca.crt' \
     EAI_REDIS_URL='rediss://eai-control-plane:<password>@redis-master.eai-data.svc:6379/0' \
     EAI_MASTER_KEY='<random-key>' EAI_OPENBAO_TOKEN='<token>' EAI_METRICS_TOKEN='<random-32-characters>'

   The keys pulled are those of controlPlane.secrets.keys (values-prod.yaml); their formats are checked by controlPlane.secretFormats (EAI_REDIS_URL starts with rediss://, EAI_DATABASE_URL has sslmode=verify-full, EAI_NATS_URL starts with tls://). Write the entries of the third-party services listed in Platform secrets.

   Expected result: kubectl get externalsecret -A shows SecretSynced for every entry.

   If it fails: kubectl describe externalsecret <name> -n <namespace> names the missing path or property.

   danger

   Never commit a secret value, never put one in a values file. Generate each one with the command of the secrets inventory (openssl rand -base64 32 for most of them).

7. Synchronize the platform layer. It installs the gateway (APISIX), LiveKit, the observability stack and the ToolHive Operator.

   argocd app sync -l eai.io/environment=prod,eai.io/layer=platform
   argocd app wait -l eai.io/environment=prod,eai.io/layer=platform --health --timeout 1800

   Expected result: the APISIX Service has an external address: kubectl -n apisix get svc.

   If it fails: an APISIX Service without address means the cluster has no load balancer (MetalLB, cloud load balancer or the one of your network team).

8. Set the values of your site. Change the example names and addresses before the first synchronization of the apps layer: <control-plane-host>, <portal-host>, <keycloak-host>, <s3-host>, <harbor-host>. The values to review are in the Helm values reference: global.imageRegistry, controlPlane.config.EAI_CORS_ORIGINS, EAI_ALLOWED_HOSTS, EAI_TRUSTED_PROXIES, EAI_OIDC_ISSUER, portal.config.*, portal.ingress.host, networkSecurity.objectStorageFQDNs.

   Check the rendering offline:

   helm lint deploy/helm/enterprise-ai -f deploy/helm/enterprise-ai/values-prod.yaml
   helm template eai deploy/helm/enterprise-ai -n enterprise-ai \
     -f deploy/helm/enterprise-ai/values-prod.yaml -f deploy/helm/enterprise-ai/values-ha.yaml > /tmp/eai.yaml

   Expected result: 1 chart(s) linted, 0 chart(s) failed, and /tmp/eai.yaml holds 38 objects with the production and HA profiles (4 Deployments: Control Plane, worker, portal, LiteLLM; the migration Job; ConfigMaps, ExternalSecrets, HPA, PDB, Cilium policies, monitors).

   If it fails: the message names the value; compare it with the reference page.

9. Synchronize the apps layer. The PreSync hook runs python -m eai.migrate before the Deployments change; the API refuses to start in prod while a production rule is broken.

   argocd app sync -l eai.io/environment=prod,eai.io/layer=apps
   argocd app wait -l eai.io/environment=prod,eai.io/layer=apps --health --timeout 1800

   Without Argo CD, on a cluster whose dependencies already run:

   helm upgrade --install enterprise-ai deploy/helm/enterprise-ai -n enterprise-ai \
     -f deploy/helm/enterprise-ai/values-prod.yaml -f deploy/helm/enterprise-ai/values-onprem-gpu.yaml

   Expected result: kubectl -n enterprise-ai get pods shows the Control Plane, worker, portal and LiteLLM pods Running, and the migration Job Completed.

   If it fails: kubectl -n enterprise-ai logs deploy/enterprise-ai-control-plane prints unsafe production settings: followed by every broken rule; fix them in the values or in OpenBao (Production rules).

10. Load the authorization model. OpenFGA needs the model of deploy/policies/openfga/model.fga; its store and model ids go to kv/eai/<env>/control-plane (EAI_OPENFGA_STORE_ID, EAI_OPENFGA_MODEL_ID).

    fga store create --name enterprise-ai --api-url https://openfga.eai-security.svc.cluster.local:8080
    fga model write --store-id <store-id> --file deploy/policies/openfga/model.fga

    Expected result: fga model list --store-id <store-id> shows one model.

    If it fails: the preshared key of OpenFGA (eai/platform/eai-security/openfga-preshared) must be given with --api-token.

Verify

Follow Verify the installation. The short check:

curl -fsS <control-plane-url>/readyz

Expected output: {"status": "ready", "checks": {"database": true, "migrations": true}, ...}.

Next steps

• TLS certificates
• First configuration
• Hosting topologies for high availability and multi-site

Troubleshooting

Symptom	Cause	Fix
Pods refused by admission with eai-verify-image-signatures	Image not signed with the Cosign key of the CI	Sign the image in Harbor, or mirror it under */mirror and sign it
Pods refused with eai-image-provenance	Image outside the signed Harbor projects without digest	Mirror the image into Harbor or reference it by digest
A data Application stays Progressing	PersistentVolumeClaim Pending	kubectl get pvc -A; create the storage class fast-ssd
ExternalSecret in SecretSyncedError	OpenBao sealed, or path missing	bao status; write the entry of the inventory
Control Plane CrashLoopBackOff with unsafe production settings	A production rule is broken	Fix each listed setting (Configuration reference)
Migration Job failed	Database unreachable or TLS refused	kubectl -n enterprise-ai logs job/<job>; check EAI_DATABASE_URL and the CA mounted for sslrootcert