io.enterpriseai.resources.EvaluationsClient
Source: sdks/java/enterprise-ai-java/src/main/java/io/enterpriseai/resources/EvaluationsClient.java. Back to the Java SDK reference index.
This page is generated by tools/docs/gen_sdk_reference.py from the SDK sources and the OpenAPI text folders. Do not edit it by hand: change the source and regenerate.
Class EvaluationsClient
Evaluation suites, runs and promotion gates.
Constructor EvaluationsClient
EvaluationsClient(Transport transport)
Creates the client on the shared transport; EnterpriseAIClient builds it and exposes it through the accessor of the same name.
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
transport | Transport | yes | Transport shared by the sub-clients of one client. |
EvaluationsClient.checkGate
JsonNode checkGate(Object body)
Check the quality gate of a target. Checks that the latest finished run of every blocking suite of the target (for target.version when given, and limited to suite_ids when given) has passed. With run_missing: true, a suite without any run is run now; otherwise it counts as missing.
Route: /v1/evaluations/gates/check (POST).
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
body | Object | yes | Target, optional suites and whether to run missing suites. |
Returns: JsonNode. Decoded JSON answer of the operation "Check the quality gate of a target".
Raises: the typed errors of the transport (Errors).
EvaluationsClient.checkGate
JsonNode checkGate(Object body, RequestOptions options)
Check the quality gate of a target. Checks that the latest finished run of every blocking suite of the target (for target.version when given, and limited to suite_ids when given) has passed. With run_missing: true, a suite without any run is run now; otherwise it counts as missing.
Route: /v1/evaluations/gates/check (POST).
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
body | Object | yes | Target, optional suites and whether to run missing suites. | |
options | RequestOptions | yes | Per call options (idempotency key, traceparent, headers, abort signal). |
Returns: JsonNode. Decoded JSON answer of the operation "Check the quality gate of a target".
Raises: the typed errors of the transport (Errors).
EvaluationsClient.createComparison
JsonNode createComparison(Object body)
Compare a candidate with a baseline. Runs one suite twice, on the baseline and on the candidate (each {type, ref, version}, type model by default, agent or prompt), and stores both runs, the per metric deltas and the verdict. Deltas are oriented by the comparator of each metric (gte: higher is better, lte: lower is better).
Route: /v1/evaluations/comparisons (POST).
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
body | Object | yes | JSON object: suite_id, baseline, candidate, optional tolerance (0 to 1, default 0) and outputs ({"baseline": [...], "candidate": [...]}, precomputed outputs per case). |
Returns: JsonNode. Decoded JSON answer of the operation "Compare a candidate with a baseline".
Raises: the typed errors of the transport (Errors).
EvaluationsClient.createComparison
JsonNode createComparison(Object body, RequestOptions options)
Compare a candidate with a baseline. Runs one suite twice, on the baseline and on the candidate (each {type, ref, version}, type model by default, agent or prompt), and stores both runs, the per metric deltas and the verdict. Deltas are oriented by the comparator of each metric (gte: higher is better, lte: lower is better).
Route: /v1/evaluations/comparisons (POST).
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
body | Object | yes | JSON object: suite_id, baseline, candidate, optional tolerance (0 to 1, default 0) and outputs ({"baseline": [...], "candidate": [...]}, precomputed outputs per case). | |
options | RequestOptions | yes | Per call options (idempotency key, traceparent, headers, abort signal). |
Returns: JsonNode. Decoded JSON answer of the operation "Compare a candidate with a baseline".
Raises: the typed errors of the transport (Errors).
EvaluationsClient.createRun
JsonNode createRun(Object body)
Run an evaluation suite. Runs a suite now and returns the finished run (201). Needs tenant_admin, developer, security_officer or ai_governance_officer or agent.
Route: /v1/evaluations/runs (POST).
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
body | Object | yes | Suite, optional target, precomputed outputs and trigger. |
Returns: JsonNode. Decoded JSON answer of the operation "Run an evaluation suite".
Raises: the typed errors of the transport (Errors).
EvaluationsClient.createRun
JsonNode createRun(Object body, RequestOptions options)
Run an evaluation suite. Runs a suite now and returns the finished run (201). Needs tenant_admin, developer, security_officer or ai_governance_officer or agent.
Route: /v1/evaluations/runs (POST).
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
body | Object | yes | Suite, optional target, precomputed outputs and trigger. | |
options | RequestOptions | yes | Per call options (idempotency key, traceparent, headers, abort signal). |
Returns: JsonNode. Decoded JSON answer of the operation "Run an evaluation suite".
Raises: the typed errors of the transport (Errors).
EvaluationsClient.createSuite
JsonNode createSuite(Object body)
Create an evaluation suite. Creates a suite. Needs tenant_admin, developer, security_officer or ai_governance_officer. Errors 400 invalid_request: target type not in model, agent, prompt, rag_pipeline, skill or tool, unknown engine or kind, comparator other than gte or lte, or, for the custom engine, a metric other than exact_match, contains, not_contains, json_valid, faithfulness, latency or refusal. A name already used answers 409 conflict. The dataset gets its size and a SHA-256 digest of the cases. Writes the audit event evaluation.suite.create.
Route: /v1/evaluations/suites (POST).
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
body | Object | yes | Name, target, engine, kind, dataset, cases, metrics and gate options. |
Returns: JsonNode. Decoded JSON answer of the operation "Create an evaluation suite".
Raises: the typed errors of the transport (Errors).
EvaluationsClient.createSuite
JsonNode createSuite(Object body, RequestOptions options)
Create an evaluation suite. Creates a suite. Needs tenant_admin, developer, security_officer or ai_governance_officer. Errors 400 invalid_request: target type not in model, agent, prompt, rag_pipeline, skill or tool, unknown engine or kind, comparator other than gte or lte, or, for the custom engine, a metric other than exact_match, contains, not_contains, json_valid, faithfulness, latency or refusal. A name already used answers 409 conflict. The dataset gets its size and a SHA-256 digest of the cases. Writes the audit event evaluation.suite.create.
Route: /v1/evaluations/suites (POST).
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
body | Object | yes | Name, target, engine, kind, dataset, cases, metrics and gate options. | |
options | RequestOptions | yes | Per call options (idempotency key, traceparent, headers, abort signal). |
Returns: JsonNode. Decoded JSON answer of the operation "Create an evaluation suite".
Raises: the typed errors of the transport (Errors).
EvaluationsClient.getComparison
JsonNode getComparison(String comparisonId)
Get an A/B comparison. Returns one comparison. Needs tenant_admin, developer, security_officer, auditor, viewer or finops. Answers 404 comparison_not_found.
Route: /v1/evaluations/comparisons/{comparison_id} (GET).
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
comparisonId | String | yes | Identifier (UUID) of the comparison. |
Returns: JsonNode. Decoded JSON answer of the operation "Get an A/B comparison".
Raises: the typed errors of the transport (Errors).
EvaluationsClient.getRun
JsonNode getRun(String id)
Get an evaluation run. Returns one evaluation run with its results per metric and per case. Open to any member of the tenant. Answers 404 not_found.
Route: /v1/evaluations/runs/{run_id} (GET).
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
id | String | yes | Identifier (UUID) of the evaluation run. |
Returns: JsonNode. Decoded JSON answer of the operation "Get an evaluation run".
Raises: the typed errors of the transport (Errors).
EvaluationsClient.getSuite
JsonNode getSuite(String id)
Get an evaluation suite. Returns one suite with its full list of cases and run_on_promotion. Open to any member of the tenant. Answers 404 not_found.
Route: /v1/evaluations/suites/{suite_id} (GET).
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
id | String | yes | Identifier (UUID) of the suite. |
Returns: JsonNode. Decoded JSON answer of the operation "Get an evaluation suite".
Raises: the typed errors of the transport (Errors).
EvaluationsClient.listComparisons
JsonNode listComparisons()
List the A/B comparisons. Lists the comparisons of the tenant, newest first, at most 200, optionally for one suite. Each gives baseline, candidate, both run ids, per metric deltas, tolerance and verdict. Needs tenant_admin, developer, security_officer, auditor, viewer or finops.
Route: /v1/evaluations/comparisons (GET).
This element takes no parameter.
Returns: JsonNode. Decoded JSON answer of the operation "List the A/B comparisons".
Raises: the typed errors of the transport (Errors).
EvaluationsClient.listComparisons
JsonNode listComparisons(Map<String, ?> query)
List the A/B comparisons. Lists the comparisons of the tenant, newest first, at most 200, optionally for one suite. Each gives baseline, candidate, both run ids, per metric deltas, tolerance and verdict. Needs tenant_admin, developer, security_officer, auditor, viewer or finops.
Route: /v1/evaluations/comparisons (GET).
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
query | Map<String, ?> | yes | Query string parameters of the route. |
Returns: JsonNode. Decoded JSON answer of the operation "List the A/B comparisons".
Raises: the typed errors of the transport (Errors).
EvaluationsClient.listRuns
Page<JsonNode> listRuns()
List the evaluation runs. Lists the evaluation runs of the tenant, newest identifier first, paginated, optionally filtered by suite and status. Each item follows the Evaluation Contract (target, engine, dataset, metrics, results, status, cost) with run_id, suite_id, trigger, case_results and error. Open to any member of the tenant.
Route: /v1/evaluations/runs (GET).
This element takes no parameter.
Returns: Page<JsonNode>. A Page: One page of a cursor paginated collection ({"items": [...], "next_cursor": "..."|null}).
Raises: the typed errors of the transport (Errors).
EvaluationsClient.listRuns
Page<JsonNode> listRuns(Map<String, ?> query)
List the evaluation runs. Lists the evaluation runs of the tenant, newest identifier first, paginated, optionally filtered by suite and status. Each item follows the Evaluation Contract (target, engine, dataset, metrics, results, status, cost) with run_id, suite_id, trigger, case_results and error. Open to any member of the tenant.
Route: /v1/evaluations/runs (GET).
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
query | Map<String, ?> | yes | Query string parameters of the route. |
Returns: Page<JsonNode>. A Page: One page of a cursor paginated collection ({"items": [...], "next_cursor": "..."|null}).
Raises: the typed errors of the transport (Errors).
EvaluationsClient.listSuites
Page<JsonNode> listSuites()
List the evaluation suites. Lists the suites of the tenant ordered by name, paginated, as Evaluation Contract documents; cases gives the number of cases. Open to any member of the tenant.
Route: /v1/evaluations/suites (GET).
This element takes no parameter.
Returns: Page<JsonNode>. A Page: One page of a cursor paginated collection ({"items": [...], "next_cursor": "..."|null}).
Raises: the typed errors of the transport (Errors).
EvaluationsClient.listSuites
Page<JsonNode> listSuites(Map<String, ?> query)
List the evaluation suites. Lists the suites of the tenant ordered by name, paginated, as Evaluation Contract documents; cases gives the number of cases. Open to any member of the tenant.
Route: /v1/evaluations/suites (GET).
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
query | Map<String, ?> | yes | Query string parameters of the route. |
Returns: Page<JsonNode>. A Page: One page of a cursor paginated collection ({"items": [...], "next_cursor": "..."|null}).
Raises: the typed errors of the transport (Errors).