B07 - Reasoning/Effort Policy

Group	Technology	Status	Service	Roadmap phase	Contracts
B - LLM management and Model Gateway	Developed	to develop	llm	3	common, model-manifest

Back to index - Français

Role and responsibilities

Maps effort none/low/medium/high/max to provider native reasoning parameters under policy.

Open source component vs developed part

Effort	Intent	OpenAI (reasoning_effort)	Anthropic / Bedrock Claude (thinking.budget_tokens)	Gemini (thinking_budget)	DeepSeek reasoner
none	No reasoning	omitted (non reasoning model) or minimal if the profile allows it	thinking disabled	0 (when the model allows it)	reject: reasoning always on
low	Short reasoning	low	1024 to 4096	1024	compatible profile always_on or reject
medium	Default	medium	8192	8192	compatible profile always_on
high	Hard tasks	high	16384 to 32000	24576	compatible profile always_on
max	Maximum allowed by policy	high (highest native)	min(model limit, policy cap)	model maximum	compatible profile always_on

Rules: the resolver either translates to the native parameter, applies an explicitly configured compatible profile, or rejects with effort_not_supported. There is never a silent conversion: the response carries the applied mapping and the X-EAI-Decision-Id. Policies can cap effort per tenant, project or purpose.

Parameters controlled by the resolver against the Model Registry capabilities: temperature, top_p (some reasoning models refuse them: rejected, not dropped silently), max_tokens (capped by max_output_tokens and policy), response_format (json_mode / json_schema), tools and tool_choice (requires tools capability), stream (requires supports_streaming), cache directives (requires supports_caching), multimodal parameters (image, audio, video inputs checked against modalities.input).

Data model

Tables (schema eai, every tenant table carries tenant_id uuid NOT NULL with an RLS policy on current_setting('app.tenant_id', true)::uuid): eai.llm_models, eai.audit_events.

API endpoints

• POST /v1/models/resolve
• POST /v1/gateway/chat/completions

POST /v1/models/resolve HTTP/1.1
Host: <control-plane-host>
X-API-Key: eai_xxx_secret
Accept-Language: en
Content-Type: application/json

{"model_id": "claude-sonnet-eu", "effort": "high", "max_tokens": 4096, "temperature": 0.2, "stream": true}

{"allowed": true, "native_params": {"thinking": {"type": "enabled", "budget_tokens": 16000}, "max_tokens": 4096}, "dropped": [], "decision_id": "dec_01H..."}

Errors use the common envelope {"error": {"code", "message", "message_i18n", "details", "request_id"}}; lists are paginated with limit and cursor.

SDK usage

Python (enterprise_ai):

from enterprise_ai import EnterpriseAI

client = EnterpriseAI(base_url="<control-plane-url>", api_key="eai_xxx_secret")
items = client.models.list()

TypeScript (@enterprise-ai/sdk):

import { EnterpriseAI } from "@enterprise-ai/sdk";

const eai = new EnterpriseAI({ baseUrl: "<control-plane-url>", apiKey: process.env.EAI_API_KEY });
const items = await eai.models.list();

Java (enterprise-ai-java):

EnterpriseAIClient client = EnterpriseAIClient.builder()
    .baseUrl("<control-plane-url>")
    .apiKey(System.getenv("EAI_API_KEY"))
    .build();
var items = client.models().list();

Go (enterprise-ai-go):

client, _ := eai.NewClient(eai.WithBaseURL("<control-plane-url>"), eai.WithAPIKey(os.Getenv("EAI_API_KEY")))
items, err := client.Models.List(ctx)

Configuration

• Environment: EAI_DATABASE_URL, EAI_OTEL_EXPORTER_OTLP_ENDPOINT.
• Helm values: controlPlane.enabled, controlPlane.replicas, controlPlane.resources, controlPlane.config, litellm.enabled, litellm.config.

Security controls

• Authentication by OIDC JWT, tenant API key with scopes, or SPIFFE mTLS; tenant resolved from the credential, never from the body.
• Authorization through OpenFGA and OPA decisions; every mutation and decision writes a hash-chained eai.audit_events row.
• Secrets never returned by the API; TLS everywhere; Cilium network policies restrict who can reach the service.

Observability

• Metrics: eai_reasoning_effort_policy_requests_total, eai_reasoning_effort_policy_errors_total, eai_reasoning_effort_policy_latency_seconds (Prometheus).
• Traces: OpenTelemetry spans eai.models.* with W3C traceparent propagation, AI spans forwarded to Opik.
• Audit actions: model.create, model.update, model.delete, model.status, policy.evaluate.

FinOps aspects

Reasoning tokens are billed as output by most providers: the effort level directly drives cost. Estimates use the reserved B_reasoning; Usage Events record reasoning_tokens separately.

Failure modes and resilience

• Stateless replicas behind the API Gateway, PostgreSQL HA for state; readiness probe GET /readyz removes unhealthy pods.
• External system unavailable: fail closed for security decisions, explicit error code, no silent bypass.

Standalone adoption (client of a client)

A client of a client can adopt this brick alone with the Python SDK sub-client client.models (TS eai.models, Java client.models(), Go client.Models). Minimal deployment: Control Plane API + PostgreSQL + Keycloak (or API keys), the Control Plane loads every domain of services/control-plane/src/eai/domains; the optional chart components are switched off in deploy/helm/enterprise-ai/values.yaml (vllm.enabled, qdrant.enabled, nats.enabled, workers.enabled).

Acceptance tests

1. A tenant cannot read or modify another tenant's resources (RLS test).
2. Every mutation produces an audit event verifiable by GET /v1/audit/verify.
3. Errors are returned in English and French according to Accept-Language.