Skip to main content

Customer installation handbook (signed release bundle)

This handbook describes how to install Enterprise AI from the signed, reproducible release bundle, including on a site without Internet access. Every command below was run on the build machine; the limits are listed at the end. French version: docs/operations/installation-handbook.md. Release notes of the same version: docs/release/en/RELEASE_NOTES-<version>.md.

1. Release tools (connected machine)​

bash tools/oci/fetch.sh                 # cosign 2.6.5, trivy, kubeconform, kyverno (sha256 verified)
bash tools/components/fetch-syft.sh     # syft 1.55.0
bash tools/release/fetch-tools.sh       # gitea 28.2.0, act_runner 5.2.4, Tekton v1.18.0, Harbor chart 1.19.4, schemas

Every version is pinned and every file is checked by sha256 (checksum published upstream, or a digest frozen in the script for files without a published checksum).

2. Build the bundle​

bash tools/release/build.sh                 # /tmp/eai-release/<version>/ and release/manifests/<version>.json
bash tools/release/verify-reproducible.sh   # two builds, every sha256 compared
bash tools/release/sign.sh                  # Syft SBOM per artefact, Cosign signatures, Trivy scans and verdict

The bundle holds: packaged Helm chart, rendered manifests (values, values-prod, values-onprem-gpu), platform manifests (Argo CD, Kyverno, KEDA, Velero, Tekton, Harbor), SQL migrations, offline docs, Python wheels and sdists (control plane and SDK), npm packs of the TypeScript SDK, Go module source archive, Java jars, navin-manager RPM and DEB packages, packs, component SBOMs, licence notices and the lock manifest.

Reproducibility: SOURCE_DATE_EPOCH (1700000000 by default), sorted tar entries, owner 0:0, fixed times, gzip headers without time or name, sources copied to a scratch directory. verify-reproducible.sh prints compared 36 artefacts, 0 differences.

Signatures: sign.sh generates a Cosign key pair under /tmp/eai-release-keys (variable EAI_SIGN_KEY_DIR, never committed) and signs manifest.json, every artefact and every SBOM (cosign sign-blob, no transparency log). In production the signing key is the customer key (HSM or OpenBao), not a test key.

Trivy verdict: sign.sh exits with an error when a HIGH or CRITICAL finding is not justified in tools/release/trivy-exceptions.yaml (fields id, justification, owner). The verdict is written to scans/gate.json.

3. Transfer and verification on the isolated site​

Copy the bundle directory (removable media), then:

bash tools/release/verify.sh /path/to/bundle                   # sha256, signatures, SBOMs and Trivy verdict
bash tools/release/verify.sh --integrity-only /path/to/bundle  # sha256, signatures and SBOMs only

The default public key is the bundle cosign.pub; pass the public key received through another channel as the second argument for an independent check.

4. Proof of installation without network​

navin-manager/server/.venv/bin/python navin-manager/packaging/offline/build-bundle.py \
    --out /tmp/eai-airgap/mirror --services opa --methods binary --skip-agent --bundle /tmp/eai-airgap/bundle.tar
bash tools/release/airgap-check.sh /tmp/eai-release/1.0.0 /tmp/eai-airgap/bundle.tar release/airgap/<date>.json

airgap-check.sh re-executes itself inside unshare -rn (empty network namespace, only lo exists), checks that a TCP connection to 1.1.1.1:443 and a DNS lookup fail, then verifies the signed bundle, verifies and extracts the navin-manager mirror, runs a binary from the mirror, installs the wheels with pip --no-index, renders the packaged chart and unpacks the SQL migrations. The evidence is written to release/airgap/<date>.json.

5. Offline validation of the cluster controls​

helm template eai deploy/helm/enterprise-ai -f deploy/helm/enterprise-ai/values-prod.yaml > /tmp/r-values-prod.yaml
KUBECONFIG=/tmp/eai-nokube/config HOME=/tmp/eai-nokube /tmp/toolchains/kyverno apply \
    deploy/policies/kyverno/cluster-policies.yaml deploy/policies/kyverno/pod-network-security.yaml \
    --resource /tmp/r-values-prod.yaml
/tmp/toolchains/kubeconform -strict -summary \
    -schema-location '/tmp/toolchains/kube-schemas/{{.ResourceKind}}{{.KindSuffix}}.json' \
    -schema-location '/tmp/toolchains/kube-schemas/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json' \
    /tmp/r-values-prod.yaml deploy/keda deploy/velero deploy/argocd deploy/cilium deploy/tekton
helm lint /tmp/toolchains/release-dl/harbor-1.19.4.tgz -f deploy/harbor/values.yaml

Important: run Kyverno with an empty KUBECONFIG. Otherwise the image signature rule tries to read its public key from the cluster of the current context.

6. Local continuous integration (Gitea and act_runner)​

bash tools/ci/gitea-local.sh   # Gitea on a free port, SQLite under /tmp, host runner, evidence release/ci-runs/<date>.json

Limits​

  • Runtime effects on a cluster (Tekton controller, Harbor, image push and pull, Kyverno admission, Cilium, KEDA, HPA, Velero backup and restore, Argo CD sync) need an acceptance cluster: not verified here.
  • The offline proof covers the bundle verification, the navin-manager mirror and the installation of the wheels without dependencies (--no-deps); third-party Python dependencies come from the navin-manager wheelhouse (--methods python), not part of this proof.
  • values-onprem-gpu.yaml uses images without digest (vllm-openai:v0.33.0), refused by the Kyverno policy eai-verify-image-signatures.