Skip to main content

G-10 Red teaming

Brick G of the Enterprise AI Platform. Technology: garak (LLM vulnerability scanner) + Promptfoo red team plugins. French version: ../../fr/bricks/G-10-red-team.md.

Role and responsibilities​

  • Probe agents and models for prompt injection, jailbreak, data leakage, toxic output, tool misuse.

Position and interactions​

  • Execution path: batch workflow, scheduled and before promotion.
  • Runs in an isolated environment with tool calls in dry run; findings become risk items (brick A).

Open source and developed parts​

  • Open source reused: garak, Promptfoo redteam.
  • Developed in the platform: Probe catalog by risk, severity mapping, regression of past findings.

Data model​

Control Plane tables (schema eai): eai.redteam_runs, eai.usage_events.

Every tenant table has tenant_id uuid NOT NULL and a row level security policy on current_setting('app.tenant_id', true)::uuid. ClickHouse usage_events is the analytics replica of eai.usage_events (ReplacingMergeTree keyed by event_id).

Contract: contracts/v1/evaluation-contract.schema.json (schema_version 1.0). Usage is always reported with contracts/v1/usage-event.schema.json, audit with contracts/v1/audit-event.schema.json.

API endpoints​

MethodPath
POST/v1/redteam/runs
GET/v1/redteam/runs/{id}
GET/v1/redteam/runs

POST /v1/redteam/runs​

Request:

{
  "target": {
    "type": "agent",
    "ref": "agt_ap",
    "version": "1.4.0"
  },
  "engine": "garak",
  "probes": [
    "promptinject",
    "dan",
    "leakreplay"
  ],
  "max_attack_success_rate": 0.05
}

Response:

{
  "id": "rtr_5",
  "status": "running"
}

GET /v1/redteam/runs/{id}​

Response:

{
  "id": "rtr_5",
  "status": "succeeded",
  "findings": [
    {
      "probe": "promptinject",
      "hit_rate": 0.02,
      "severity": "medium"
    }
  ]
}

GET /v1/redteam/runs​

Response:

{
  "items": [
    {
      "id": "rtr_5",
      "status": "succeeded"
    }
  ],
  "next_cursor": null
}

SDK examples​

Python​

from enterprise_ai import EnterpriseAI

client = EnterpriseAI(base_url="<control-plane-url>", api_key="eai_live_xxx", tenant="acme", locale="en")
client.evaluation.redteam(target={"type": "agent", "ref": "agt_ap"}, engine="garak", probes=["promptinject", "dan"])
client.evaluation.get_redteam_run(run_id="rtr_5")
client.evaluation.list_redteam_runs(limit=20)

TypeScript​

import { EnterpriseAI } from "@enterprise-ai/sdk";

const client = new EnterpriseAI({ baseUrl: "<control-plane-url>", apiKey: process.env.EAI_API_KEY, tenant: "acme", locale: "en" });
await client.redteam.createRun({ target: { type: "agent", ref: "agt_ap" }, engine: "garak", probes: ["promptinject", "dan"] });
await client.redteam.getRun({ runId: "rtr_5" });
await client.redteam.listRuns({ limit: 20 });

Java​

EnterpriseAIClient client = EnterpriseAIClient.builder()
    .baseUrl("<control-plane-url>")
    .apiKey(System.getenv("EAI_API_KEY"))
    .tenant("acme")
    .locale(Locale.ENGLISH)
    .build();
client.redteam().createRun(Map.of("target", Map.of("type", "agent", "ref", "agt_ap"), "engine", "garak", "probes", List.of("promptinject", "dan")));
client.redteam().getRun("rtr_5");
client.redteam().listRuns(Map.of("limit", 20));

Go​

client, err := eai.NewClient(eai.WithBaseURL("<control-plane-url>"), eai.WithAPIKey(os.Getenv("EAI_API_KEY")), eai.WithTenant("acme"), eai.WithLocale("en"))
if err != nil { log.Fatal(err) }
ctx := context.Background()
_, err = client.RedTeam.CreateRun(ctx, map[string]any{"target": map[string]any{"type": "agent", "ref": "agt_ap"}, "engine": "garak", "probes": []string{"promptinject", "dan"}})
_, err = client.RedTeam.GetRun(ctx, "rtr_5")
_, err = client.RedTeam.ListRuns(ctx, eai.WithLimit(20))

Configuration​

Environment variables: EAI_PROMPTFOO_NPX.

Helm values keys (deploy/helm/enterprise-ai/values.yaml): workers.items[name=redteam].

Security​

  • Red team traffic tagged and excluded from production analytics; attack payloads stored encrypted.
  • Tenant resolved from the credential, never from the body; PostgreSQL row level security on every tenant table.
  • Every mutation and policy decision writes a hash chained audit event (G-11).
  • Errors use the standard envelope with message_i18n en and fr.

Observability​

  • Hit rate per probe and per version.
  • W3C traceparent propagated, X-Request-Id returned, spans carry eai.tenant_id, eai.project, eai.agent_id, eai.run_id.

FinOps units​

  • Usage Events with labels.purpose=redteam, budgeted by a dedicated quota.

Failure modes and resilience​

  • Target unreachable -> run failed, gate blocked.

Standalone adoption​

Security teams use it alone against any OpenAI compatible endpoint registered as a provider.

Acceptance tests​

  1. promptinject probe hit rate above threshold blocks promotion.

Implementation roadmap