Skip to main content

Troubleshooting

SymptomCauseFix
navin-manager serve exits with a list of problemsA production setting is unsafe (TLS, client certificates, cookie flag, metrics token, scrypt cost, master key, sslmode).Fix each line printed by navin-manager check.
master key file ... must not be readable by group or othersThe key file has a group or other permission.chown navin-manager:navin-manager /etc/navin-manager/master.key && chmod 0600 /etc/navin-manager/master.key
sealed value was altered or sealed with another master keyThe master key changed after secrets were stored.Restore the original key from the bank vault. A new key cannot read the old secrets.
/readyz answers 503 database_unavailableThe manager database is unreachable or refuses TLS.Check database_url, the CA file in sslrootcert, the firewall to PostgreSQL, the password in server.env.
The console answers 503 ui_not_builtThe console files are missing in NM_UI_DIR.Reinstall the package, or in a checkout run npm ci && npm run build in navin-manager/ui.
nm-agent register fails with bootstrap_token_invalidThe token expired, was revoked, used up, or typed wrongly.Create a new token in Hosts, Add hosts.
nm-agent register refuses the CA fingerprintThe --ca-sha256 value does not match the CA served by the manager (wrong server or interception).Copy the command again from the console; check that the agent reaches the right manager.
host_in_other_clusterThe host is still registered in another cluster.Decommission it there first.
Agent requests refused with agent_signature_invalidThe clock of the host is more than 5 minutes away from the manager, or the agent secret was replaced.Synchronize the clock (chronyd or systemd-timesyncd); register the host again if it was re-registered elsewhere.
Host shown as lostNo heartbeat for 120 seconds: agent stopped, network cut, port 8461 filtered.systemctl status navin-manager-agent, journalctl -u navin-manager-agent, test curl https://<manager>:8461/healthz from the host. The alert resolves at the next heartbeat.
Preflight time_sync failedNo time synchronization service is active.Enable chronyd (RHEL) or systemd-timesyncd (Ubuntu).
Install refused with artifact_missingThe mirror lacks an artifact, image or wheelhouse of the requested version.Build a bundle with that service and version, unpack it, reload the mirror. Administration, Mirror lists what is missing.
Task failed with checksum_mismatchA downloaded file differs from the manifest (corrupted mirror or proxy rewrite).Run verify-bundle.py --mirror; restore the bundle. Never edit manifest.json by hand.
Task failed with path_not_allowedThe task targets a path outside the managed directories, or a symbolic link is in the path.Remove the symbolic link (for example a /var/lib/navin link to another disk: mount the disk there instead).
Task failed with not_rootA rootless agent (sandbox) received a task that needs root.Use a production host with the agent running as root, or a sandbox cluster.
Task timed_outNo result before the timeout: agent restarted, command blocked.Read the task logs, then Retry. A task interrupted by an agent restart fails with agent_restarted.
Preflight port_free failedAnother process listens on the port of the component.Stop that process or change the port parameter of the service.
health_check_failed after startThe component did not become healthy.Logs on the component, then fix its configuration and Restart.
A user cannot sign in after losing the TOTP deviceA security_admin resets the second factor in Administration, Users; the user enrolls again.
Every administrator is locked outnavin-manager create-admin --username <name> --password-stdin on the manager host resets that local account.
GET /api/v1/audit/verify reports valid: falseAn audit event was altered in the database.Keep the database as evidence and raise a security incident: the response gives the first altered event.