Skip to main content

G-03 AI tracing (prompts, responses, tools, scores)

Brick G of the Enterprise AI Platform. Technology: Opik (Apache-2.0, self hosted) for generations, prompts, tool calls, sessions and scores. French version: ../../fr/bricks/G-03-ai-tracing.md.

Role and responsibilities​

  • Store prompts, responses, tool inputs/outputs, prompt versions, latency and token usage per generation.
  • Attach scores (user feedback, Ragas metrics, LLM as judge) to traces for quality analysis.

Position and interactions​

  • Execution path: event (async export from gateway and runtime).
  • The OTel trace id is kept in the Opik trace metadata (eai.trace_id); GET /v1/telemetry/traces/{trace_id} returns links.opik (project page and metadata key).

Open source and developed parts​

  • Open source reused: Opik, Opik OTLP and REST API, OTel integration.
  • Developed in the platform: Content capture policy per classification, redaction before export, project mapping tenant -> Opik project (tenant_id in the metadata).

Data model​

Control Plane tables (schema eai): eai.telemetry_spans, eai.eval_runs.

Every tenant table has tenant_id uuid NOT NULL and a row level security policy on current_setting('app.tenant_id', true)::uuid. ClickHouse usage_events is the analytics replica of eai.usage_events (ReplacingMergeTree keyed by event_id).

Contract: contracts/v1/evaluation-contract.schema.json (schema_version 1.0). Usage is always reported with contracts/v1/usage-event.schema.json, audit with contracts/v1/audit-event.schema.json.

API endpoints​

MethodPath
POST/v1/telemetry/spans
GET/v1/telemetry/traces/{trace_id}

POST /v1/telemetry/spans​

Request:

{
  "spans": [
    {
      "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
      "span_id": "00f067aa0ba902b7",
      "name": "gen_ai.chat",
      "kind": "client",
      "start_time": "2026-10-29T10:00:00.120Z",
      "end_time": "2026-10-29T10:00:01.940Z",
      "attributes": {
        "gen_ai.request.model": "gpt-4o-mini",
        "gen_ai.usage.input_tokens": 812,
        "eai.agent_id": "agt_ap",
        "eai.run_id": "run_42"
      }
    }
  ]
}

Response:

{
  "accepted": 1,
  "rejected": 0
}

GET /v1/telemetry/traces/{trace_id}​

Response:

{
  "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
  "spans": [
    {
      "span_id": "00f067aa0ba902b7",
      "name": "gen_ai.chat",
      "duration_ms": 1820
    }
  ],
  "links": {
    "opik": {
      "url": "https://<opik-host>/default/redirect/projects?name=eai-platform",
      "metadata_key": "eai.trace_id",
      "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736"
    }
  }
}

SDK examples​

Python​

from enterprise_ai import EnterpriseAI

client = EnterpriseAI(base_url="<control-plane-url>", api_key="eai_live_xxx", tenant="acme", locale="en")
client.telemetry.ingest(spans=spans)
client.telemetry.trace(trace_id="4bf92f3577b34da6a3ce929d0e0e4736")

TypeScript​

import { EnterpriseAI } from "@enterprise-ai/sdk";

const client = new EnterpriseAI({ baseUrl: "<control-plane-url>", apiKey: process.env.EAI_API_KEY, tenant: "acme", locale: "en" });
await client.telemetry.ingestSpans({ spans: spans });
await client.telemetry.trace({ traceId: "4bf92f3577b34da6a3ce929d0e0e4736" });

Java​

EnterpriseAIClient client = EnterpriseAIClient.builder()
    .baseUrl("<control-plane-url>")
    .apiKey(System.getenv("EAI_API_KEY"))
    .tenant("acme")
    .locale(Locale.ENGLISH)
    .build();
client.telemetry().ingestSpans(Map.of("spans", spans));
client.telemetry().trace("4bf92f3577b34da6a3ce929d0e0e4736");

Go​

client, err := eai.NewClient(eai.WithBaseURL("<control-plane-url>"), eai.WithAPIKey(os.Getenv("EAI_API_KEY")), eai.WithTenant("acme"), eai.WithLocale("en"))
if err != nil { log.Fatal(err) }
ctx := context.Background()
_, err = client.Telemetry.IngestSpans(ctx, map[string]any{"spans": spans})
_, err = client.Telemetry.Trace(ctx, "4bf92f3577b34da6a3ce929d0e0e4736")

Configuration​

Environment variables: EAI_OPIK_URL, EAI_OPIK_UI_URL, EAI_OPIK_WORKSPACE, EAI_OPIK_PROJECT, EAI_OPIK_API_KEY.

Helm values keys (deploy/helm/enterprise-ai/values.yaml): controlPlane.config.EAI_OPIK_URL, controlPlane.secrets.

Security​

  • Content capture off for secret data; confidential data captured only after DLP masking.
  • Tenant resolved from the credential, never from the body; PostgreSQL row level security on every tenant table.
  • Every mutation and policy decision writes a hash chained audit event (G-11).
  • Errors use the standard envelope with message_i18n en and fr.

Observability​

  • Export lag and dropped generations.
  • W3C traceparent propagated, X-Request-Id returned, spans carry eai.tenant_id, eai.project, eai.agent_id, eai.run_id.

FinOps units​

  • Token usage in Opik is a view, not the accounting source: costs come from H-02.

Failure modes and resilience​

  • Opik unavailable -> in-process fallback buffers, business calls continue.

Standalone adoption​

Teams can use Opik through the platform for their own prompts without the agent runtime.

Acceptance tests​

  1. A chat completion appears in Opik with prompt version and token usage.
  2. A secret classified request has no content in Opik.

Implementation roadmap