Skip to main content

Gateway playground

The Gateway playground screen lets you call the LLM gateway of the Control Plane like an application does, with streaming, a reasoning effort and a data classification, and inspect the decision id and the estimated and measured cost of each call. Portal path: /llm/playground. It is used by developers and AI platform engineers who test a model through the same policies as production. The screen is built in portal/src/screens/llm-tools.tsx (Playground), with the request body of chatBody in portal/src/lib/request-bodies.ts.

What the screen shows​

The column Settings:

FieldContent
ModelSelector filled from GET /v1/models (first model by default). If the list cannot be read, a text field Model id; without model, No model registered.
Effortnone, low, medium, high, max (default none).
Data classificationpublic, internal, confidential, restricted (default internal).
Streaming (SSE)On by default: the answer arrives as server-sent events.
System promptOptional system message.
Estimate costButton, with the JSON estimate below.

The column Conversation:

  • three badges: Decision (X-EAI-Decision-Id header or stream event), Estimated (X-EAI-Cost-Estimated) and Measured (X-EAI-Cost-Measured, or the cost of the answer when streaming is off), - before the first call;
  • the messages, yours on the right, the answers on the left, filled as the stream arrives; "Send a message to call the gateway." when empty;
  • the error of the last call, if any;
  • the field Write a message with Send, replaced by Stop during a call;
  • Clear when the conversation has messages.

Every call is sent with purpose: "playground", the chosen effort and the chosen classification in the eai extension of the OpenAI body.

Actions​

Button or actionWhat it doesAPI callRoles
Send (or Enter)Sends the history with the new message; streams the answer when Streaming (SSE) is on.POST /v1/gateway/chat/completionscredential with the scope models:invoke
StopAborts the running call in the browser.noneeveryone
Estimate costEstimates the cost of the conversation plus the message being typed, without calling the model.POST /v1/gateway/estimatecredential with the scope models:invoke
ClearEmpties the conversation and the badges.noneeveryone
Model selectorLists the registered models.GET /v1/modelsLLM readers

Roles​

  • The gateway routes check a scope, not a role: the credential must carry models:invoke (require_scope("models:invoke") in services/control-plane/src/eai/domains/llm/__init__.py). A portal session opened with the openid scope carries every scope.
  • The model list needs one of the LLM readers roles (ROLES.llmReaders of portal/src/lib/kit.ts).
  • The gateway then applies the policies of the tenant: model status and kill switches, allowlists, region, data classification against the model and against your clearance, DLP, isolation mode and budget.
  • The call needs a tenant: a principal without tenant gets platform_only.

Typical tasks​

Test a model through the gateway​

  1. Choose the settings. Pick the Model, keep Effort none and Data classification internal, so that the call goes through the usual policies. Expected result: the Send button is active once you type a message. If it fails: No model registered means the registry is empty; see the Models registry.
  2. Send. Type a message in Write a message and click Send. Expected result: the answer streams in; Decision, Estimated and Measured are filled. If it fails: the error is shown above the field (see the Troubleshooting section).
  3. Follow the decision. Copy the Decision id, so that you can find the call in the traces and the audit log. Expected result: a decision id in the badge. If it fails: - means the call failed before the decision.

The same call from the command line (see docs/sdks/cli-reference.md):

enterprise-ai chat -m <model-id> --effort low "Hello"

<model-id>: registry id of the model.

Estimate before sending a long prompt​

  1. Prepare the message. Type the System prompt and the message, without sending it. Expected result: the text is in the field. If it fails: nothing to fix, the estimate needs a model only.
  2. Estimate. Click Estimate cost. Expected result: a JSON estimate under the button. If it fails: "The estimated cost exceeds the remaining budget." (budget_exceeded, 429) means the budget of the tenant or project is spent.

Troubleshooting​

SymptomCauseFix
insufficient_scope (403)The credential has no models:invoke scope.Use a credential that carries models:invoke.
"This model is disabled (kill switch)." (model_disabled, 403) or "This provider is disabled (kill switch)." (provider_disabled, 403)A kill switch is on.Enable the model in the Models registry or the provider in Providers.
"This model is not approved for use (status <status>)." (model_blocked, 403)Status other than approved.Approve the model, or pick another one.
"This model is not in the allowlist." (model_not_allowed, 403)The tenant or project allowlist excludes the model.Pick an allowed model.
"Data classification <classification> is above what this model accepts." (classification_exceeded, 403)Data classification above Max classification of the model.Lower the classification or pick another model.
"Data classification <classification> is above the caller clearance." (clearance_exceeded, 403)Your clearance is lower than the classification.Lower the classification.
"Sensitive data was detected and the request was blocked." (dlp_blocked, 422)The DLP found sensitive data in the message or the answer.Remove the data from the message.
"Mode <mode>: model <model_id> of provider <provider> is external and cannot be routed." (isolation_external_route, 403)The tenant or project is in local_only or air_gapped.Use a local model, or change the Isolation mode.
"The estimated cost exceeds the remaining budget." (budget_exceeded, 429)Budget spent.Raise the budget or wait for the next period.
"Model calls are disabled for this tenant." (tenant_llm_disabled, 403)The tenant kill switch is on.Ask a security officer.