Skip to main content

ADR-0150: Internal operation engine of the Platform Manager instead of Temporal

Context​

The architecture (A3.1, Workflow Engine) asks for long operations with resume, validation and rollback, and names Temporal. The Navin Platform Manager is installed before the platform (bootstrap, recovery while the Control Plane is down, isolated sites without Internet). It must work with one server process and its PostgreSQL database.

Decision​

The Manager keeps its internal engine (navin_manager/engine/core.py). Every operation, stage, task and attempt is a PostgreSQL row (nm.operations, nm.stages, nm.tasks): the server process keeps no state in memory. An engine tick reads the durable state, queues the next stage, applies timeouts, attempts, the failure budget of a stage and aborts. A server restart therefore resumes the operation where the database left it. Validation runs through the service checks and smoke tests planned as stages; rollback is an operation of the engine (upgrade rollback, restore). Two person approvals, operating modes, the security lock of the cluster and the profile rules are checked when the operation is created.

Temporal stays available to the agents of the platform (adapter runtime/temporal.py, ADR-0102). The Manager does not depend on it: a Temporal cluster would be one more component to install, back up and repair before the Manager could repair anything.

Consequences​

One component to run for the Manager; the state of the operations is backed up with the Manager database. Workflows are not deterministic replayed code as in Temporal: each stage is idempotent and described by typed tasks.

Evidence​

  • navin-manager/server/tests/test_engine_resume.py::test_operation_resumes_after_a_server_restart: a second Python process, with no state in memory, resumes the operation.
  • navin-manager/server/tests/test_foundation.py::test_engine_failure_retry_abort_and_timeout: attempts, abort, timeout.
  • navin-manager/server/tests/test_lifecycle_upgrades.py::test_stateless_upgrade_with_canary_then_rollback: rollback.