Skip to main content

Quality gates

The Quality gates screen lets you check whether a target (an agent, a model, a prompt and so on) passes every blocking evaluation suite before you promote it to another environment, and read the evidence of the check. Portal path: /quality/gates. Release managers, developers and AI governance officers use it before a promotion; CI pipelines call the same route. The screen is drawn by portal/src/screens/quality.tsx.

What the screen shows​

The page holds one tester, Check a gate ("Returns pass or block with the evaluation evidence."):

  • Target (required, type:ref@version, example agent:invoice-assistant@1.2.0; without a type the target is an agent).
  • Suites (optional, comma separated suite ids; empty means every blocking suite of the target).
  • Run missing evaluations (default off): run a suite that has no finished run for this version.
  • The button Run and a Result area. On success the area shows the badge Gate passed and the JSON of the check (passed, target, checks). On a block it shows the error gate_failed with the failing suites in its details.

Actions​

Button or actionWhat it doesAPI callRoles
RunFor each blocking suite of the target (or of Suites), takes the last finished run for this version, runs it when Run missing evaluations is on and none exists, then passes only when every suite passed. The check is written to the audit log.POST /v1/evaluations/gates/checkAny tenant member

The form sends this body (the target is parsed by portal/src/lib/mappers.ts):

{"target": {"type": "agent", "ref": "invoice-assistant", "version": "1.2.0"}, "suite_ids": ["<suite-id>"], "run_missing": false}

<suite-id>: evaluation_id of a suite from the Evaluation suites screen. Each entry of checks gives suite_id, suite, run_id, status (passed, failed, error or missing) and results.

tip

The same suites run by themselves at each promotion: when an approver approves a promotion of an agent or a model, the Control Plane runs the promotion suites of the target and applies the new status only when the blocking ones pass.

Roles​

The portal sets no role on this tester and the route POST /v1/evaluations/gates/check only needs a tenant (services/control-plane/src/eai/domains/evaluation/__init__.py). With Run missing evaluations on, the missing suites are run on behalf of the caller, who must be able to reach the target.

Typical tasks​

Check a version before its promotion​

  1. Type the target. The gate looks at the runs of this exact version. Fill Target with agent:<name>@<version>. Expected result: the form is ready. If it fails: invalid_request (400) with target.type means the type is not supported.
  2. Run the check. Only finished runs count. Click Run. Expected result: Gate passed and a checks list where every suite is passed. If it fails: the error gate_failed (409) lists the failures; a suite in missing has no run for this version.
  3. Fill the gaps. A missing run blocks the gate. Turn Run missing evaluations on and click Run again, or start the runs from Evaluation runs. Expected result: the missing suites now have a status. If it fails: a suite in error could not reach the target; check the target or provide outputs in a manual run.

Check only some suites​

  1. List the suite ids. A release may need a subset of the suites. Type the ids in Suites, separated by commas, then click Run. Expected result: checks holds only these suites. If it fails: a suite that is not blocking or not attached to the target is ignored; check its Blocking gate and Target.

Troubleshooting​

SymptomCauseFix
gate_failed (409): "Quality gate failed: promotion is blocked."At least one blocking suite is failed, error or missing.Read failures in the details, fix the target and run the suites again.
invalid_request (400) with target.typeThe target type is not one of model, agent, prompt, rag_pipeline, skill, tool.Fix the type in Target.
Gate passed with an empty checksNo blocking suite matches the target.Create a blocking suite for this target on the Evaluation suites screen.
A suite stays missingNo finished run exists for this version.Turn Run missing evaluations on, or run the suite by hand.