Operations reference
Every action of the console (install, start, upgrade, backup, decommission and the others) is an operation made of ordered stages; a stage runs its tasks in parallel, one per host, and the next stage starts when every task of the stage has finished (CONTRACT.md section 11). This page lists the operation types, the stage names and the task types. It is generated by navin-manager/docs/tools/generate_reference.py from navin_manager/engine/core.py, navin_manager/services/planner.py, navin_manager/services/tasks.py, the locales and the agent handlers: do not edit it by hand.
Operation types
The type is shown in the Operations page and stored in nm.operations.type. The stages column lists the stages the planner can add, in their order; a stage without tasks is not created.
| Operation type | Description | Stages |
|---|---|---|
service.install | Install one or more services (install wizard), dependencies first; also used by Reinstall. | provision the bindings; dependencies; preflight checks; node prerequisites (podman, Python, tar); users and directories; artifacts and images; configuration files and certificates; systemd units; start <components>; health checks; service check |
blueprint.apply | Install the services of a blueprint (a whole topology described in one document); same stages as service.install. | provision the bindings; dependencies; preflight checks; node prerequisites (podman, Python, tar); users and directories; artifacts and images; configuration files and certificates; systemd units; start <components>; health checks; service check |
service.action | Start, stop, restart or rolling restart of components; also the failover of a service and the reconcile of a drift finding. | start <components>; stop <components>; restart <components>; restart batch <batch> of <batches>; health checks of batch <batch> of <batches>; health checks |
service.config_apply | Render and write every configuration file, then restart the stale instances (rolling, all or none) and run the health checks. | configuration files and certificates; configuration of the peers; restart <components>; restart batch <batch> of <batches>; health checks of batch <batch> of <batches>; health checks |
service.command | Run a custom command of the catalog (its steps: command.run, health.check, service.control, file.write, logs.tail). | command <command> |
service.components | Add a component on hosts, or remove one instance. | preflight checks; node prerequisites (podman, Python, tar); users and directories; artifacts and images; configuration files and certificates; systemd units; start <components>; health checks; stop <components>; remove the units; remove the records |
service.uninstall | Stop the service, remove its units, optionally purge its data, then remove its records. | stop <components>; remove the units; purge the data; remove the records |
service.logs | Read the last lines of the log of an instance (logs.tail). | read the logs |
service.mode | Change the operating mode of a service (maintenance and the other modes of navin-manager/server/src/navin_manager/services/modes.py). | see the description |
service.upgrade | Upgrade a service to a new release: canary host first, then host by host, migrations, new version recorded. | canary <host>, new release; canary <host>, stop; canary <host>, files and units; canary <host>, start; canary <host>, health checks; canary <host>, smoke test; canary <host> upgraded; <host>, new release; <host>, stop; <host>, files and units; <host>, start; <host>, health checks; <host> upgraded; migration <migration>; new version recorded |
service.rollback | Bring an upgraded service back to its previous release. | stop every instance; <host>, previous release; <host>, previous files and units; <host>, start; <host>, health checks; <host> back on the previous release; previous version recorded |
service.restore | Bring an upgraded service back to its previous release and restore the backup taken before the upgrade. | see the description |
backup.create | Back up the data of services: directory, backup command, digests of the files, record. | backup directory; backup; digests of the backup files; backup recorded |
backup.restore | Restore a backup: integrity check, stop, restore, start, verification of the data. | digests of the backup files; integrity of the backup; stop before the restore; restore; start after the restore; verification of the restored data; restore recorded |
drift.scan | Compare the digests of the managed files with the expected ones (drift detection). | Digests of the managed files |
network.firewall_members | Send the members of the cluster and the restricted ports to the host firewall of every host (job network.firewall_members, when a host joins, leaves or moves). | Firewall of the nodes: ports between nodes open to the cluster only |
support.bundle | Collect the log tails of the instances and build a sanitized support archive. | Log tails of the instances; Sanitized archive |
acceptance.run | Installation acceptance: preflight, liveness, readiness, functional smoke tests, signed report. | Acceptance: preflight; Acceptance: liveness of the instances; Acceptance: readiness of the services; Acceptance: functional smoke tests; Acceptance: signed report |
host.preflight | Collect the facts of a host again (facts.collect) and evaluate the preflight. | see the description |
host.facts | Refresh the facts of a host (facts.collect). | see the description |
host.parameters | Apply the system parameters of a host (kernel parameters, limits). | see the description |
host.decommission | Decommission a host without components: the agent stops and forgets its identity, then the server records the host as decommissioned. | see the description |
host.ssh_bootstrap | Install and register the agent on hosts over SSH (fingerprint of each host key checked, key kept in memory only). | see the description |
host.node_bootstrap | Prepare a node over SSH (node security: service account, sudoers, agent) before its registration. | see the description |
pki.rotate | Issue new certificates for the instances of a service and restart them. | see the description |
mcp.hosting.apply | Apply the hosting of the MCP servers on the mcp-host service. | see the description |
Stage names
Every stage name of the locales, as the console shows it. <service>, <host> and the other values in angle brackets are filled at planning time.
| Key | Name |
|---|---|
hosts.stage.agent | Stop the agent |
hosts.stage.cert_install | Install the certificates |
hosts.stage.restart | Restart the components |
hosts.stage.revoke | Revoke the certificates |
hosts.stage.ssh_bootstrap_summary | Verdict of the connection |
lifecycle.stage.acceptance_liveness | Acceptance: liveness of the instances |
lifecycle.stage.acceptance_preflight | Acceptance: preflight |
lifecycle.stage.acceptance_readiness | Acceptance: readiness of the services |
lifecycle.stage.acceptance_report | Acceptance: signed report |
lifecycle.stage.acceptance_smoke | Acceptance: functional smoke tests |
lifecycle.stage.bundle_build | Sanitized archive |
lifecycle.stage.bundle_logs | Log tails of the instances |
lifecycle.stage.drift_scan | Digests of the managed files |
network.stage.firewall_members | Firewall of the nodes: ports between nodes open to the cluster only |
services.stage.backup_digest | <service>: digests of the backup files |
services.stage.backup_done | <service>: backup recorded |
services.stage.backup_prepare | <service>: backup directory |
services.stage.backup_run | <service>: backup |
services.stage.command | <service>: command <command> |
services.stage.configure | <service>: configuration files and certificates |
services.stage.dependencies | <service>: dependencies |
services.stage.fetch | <service>: artifacts and images |
services.stage.forget | <service>: remove the records |
services.stage.health | <service>: health checks |
services.stage.logs | <service>: read the logs |
services.stage.nothing | <service>: record |
services.stage.peers | <service>: configuration of the peers |
services.stage.preflight | <service>: preflight checks |
services.stage.prepare | <service>: users and directories |
services.stage.prerequisites | <service>: node prerequisites (podman, Python, tar) |
services.stage.provision | <service>: provision the bindings |
services.stage.purge | <service>: purge the data |
services.stage.remove_units | <service>: remove the units |
services.stage.restart | <service>: restart <components> |
services.stage.restore_check | <service>: integrity of the backup |
services.stage.restore_digest | <service>: digests of the backup files |
services.stage.restore_done | <service>: restore recorded |
services.stage.restore_run | <service>: restore |
services.stage.restore_start | <service>: start after the restore |
services.stage.restore_stop | <service>: stop before the restore |
services.stage.restore_verify | <service>: verification of the restored data |
services.stage.rollback_configure | <service>: <host>, previous files and units |
services.stage.rollback_done | <service>: previous version recorded |
services.stage.rollback_fetch | <service>: <host>, previous release |
services.stage.rollback_health | <service>: <host>, health checks |
services.stage.rollback_host_done | <service>: <host> back on the previous release |
services.stage.rollback_start | <service>: <host>, start |
services.stage.rollback_stop | <service>: stop every instance |
services.stage.rolling_batch | <service>: restart batch <batch> of <batches> |
services.stage.rolling_health | <service>: health checks of batch <batch> of <batches> |
services.stage.service_check | <service>: service check |
services.stage.smoke | <service>: functional smoke test |
services.stage.start | <service>: start <components> |
services.stage.stop | <service>: stop <components> |
services.stage.units | <service>: systemd units |
services.stage.upgrade_canary_configure | <service>: canary <host>, files and units |
services.stage.upgrade_canary_done | <service>: canary <host> upgraded |
services.stage.upgrade_canary_fetch | <service>: canary <host>, new release |
services.stage.upgrade_canary_health | <service>: canary <host>, health checks |
services.stage.upgrade_canary_smoke | <service>: canary <host>, smoke test |
services.stage.upgrade_canary_start | <service>: canary <host>, start |
services.stage.upgrade_canary_stop | <service>: canary <host>, stop |
services.stage.upgrade_done | <service>: new version recorded |
services.stage.upgrade_host_configure | <service>: <host>, files and units |
services.stage.upgrade_host_done | <service>: <host> upgraded |
services.stage.upgrade_host_fetch | <service>: <host>, new release |
services.stage.upgrade_host_health | <service>: <host>, health checks |
services.stage.upgrade_host_start | <service>: <host>, start |
services.stage.upgrade_host_stop | <service>: <host>, stop |
services.stage.upgrade_migration | <service>: migration <migration> |
Task types run by the agent
These tasks run on a host, one at a time, by the Navin agent (Python nm_agent/handlers.py and Go agent-go/internal/agent/handlers.go). Their payload is sealed in the database; the console shows only the summary.
| Task type | Description |
|---|---|
agent.config | Update agent.json from the Manager. |
agent.decommission | Stop the agent and forget the identity of the host. |
archive.extract | Extract an archive; absolute names, .., devices, FIFOs and escaping links are refused (archive_unsafe). |
artifact.fetch | Download an artifact from the mirror and check its sha256 before it reaches its destination (checksum_mismatch otherwise). |
cert.install | Write the certificate, key and CA of an instance. |
command.run | Run an allowed absolute program from its argument list, never through a shell. |
dir.ensure | Create a directory with its owner, group and mode. |
facts.collect | Collect the facts of the host. |
file.remove | Remove a managed file. |
file.write | Write a file with its owner, group and mode; a sensitive content is masked in the logs. |
files.digest | Compute the sha256 of managed files (drift and backups). |
firewall.open | Open the client ports of a component to every source and the ports only the cluster uses to the hosts of the cluster and the Manager (firewalld ipset and rich rules, or ufw rules per member). |
health.check | Run the health checks (http, tcp, command) with their retries. |
image.load | Load a container image from the mirror and check its digest; with store, into the read-only image store <store>/storage that a rootless service user lists. |
image.pull | Pull a container image from the registry and check its digest; with store, into the read-only image store <store>/storage that a rootless service user lists. |
limits.apply | Write a limits file in /etc/security/limits.d. |
link.ensure | Point a symbolic link (current, venv) to a release directory; with displace_dir a real directory in the way is kept as <link>.nm-legacy. |
logs.tail | Read the last lines of a journal or of a log file. |
packages.install | Install system packages with dnf, apt or pacman, at the version pinned by the catalog, repository signatures always checked. |
preflight.run | Run the preflight checks of a service on the host. |
python.venv | Create the virtual environment of one release from the wheelhouse of the mirror, with pip --require-hashes --no-index. |
quadlet.unit | Write the Quadlet files of a Podman component (navin-*.container, navin-*.network in /etc/containers/systemd), disable and remove the systemd unit they replace, then reload systemd. |
service.control | Start, stop, restart, enable or disable a component (systemd or process driver). |
smoke.run | Run the functional smoke test of a service. |
sysctl.apply | Write and apply a file in /etc/sysctl.d. |
systemd.mask | Stop, disable and mask the systemd units a distribution package starts on its own (distribution_units of the catalog, for example asterisk.service), only when each unit file belongs to a package of the install list; unmask only at uninstall, never start. |
systemd.unit | Write a systemd unit and reload systemd. |
user.ensure | Create the system account of a service; with subordinate_ids, give it subordinate uid and gid ranges of 65536 ids (usermod) for rootless Podman. |
Task types run by the server
These tasks have no host: the engine runs them inside the Manager (decorator @server_task).
| Task type | Description |
|---|---|
server.host_decommissioned | Record the host as decommissioned once its agent stopped. |
server.lifecycle.acceptance_preflight | Preflight stage of the installation acceptance. |
server.lifecycle.acceptance_report | Build and sign the acceptance report. |
server.lifecycle.backup_check | Check the digests of a backup before a restore. |
server.lifecycle.backup_done | Record a finished backup. |
server.lifecycle.instances_version | Record the version now running on the upgraded instances. |
server.lifecycle.restore_done | Record a finished restore. |
server.lifecycle.rollback_done | Record the previous version after a rollback. |
server.lifecycle.support_bundle | Build the sanitized support archive. |
server.lifecycle.upgrade_done | Record the new version after an upgrade. |
server.node_bootstrap | Prepare a node over SSH (runs in the background). |
server.services.dependencies | Check the dependencies of the services of the operation. |
server.services.forget | Remove the records of an uninstalled service. |
server.services.preflight | Evaluate the preflight checks of the planned instances. |
server.services.register | Record an operation that needs no host task. |
server.services.remove_instance | Remove the record of one instance. |
server.ssh_bootstrap | Install and register the agent over SSH (runs in the background). |
server.ssh_bootstrap_summary | Verdict of a multi-machine SSH bootstrap: fails the operation with ssh_bootstrap_partial when a machine was not connected, lists the machines connected otherwise. |
Task statuses
Statuses of a task (nm.tasks.status). An operation is pending, running, succeeded, failed or aborted; a stage is pending, running, succeeded, failed, aborted or skipped.
| Status | Description |
|---|---|
waiting | Waiting for the previous stages. |
queued | Ready, waiting for its agent to claim it. |
running | Claimed and running. |
succeeded | Finished without error. |
failed | Finished with an error. |
timed_out | Not finished within its timeout. |
aborted | Stopped by an abort of the operation. |
skipped | Not run (operation stopped before it). |