F-11 Multimodal DLP

Brick F of the Enterprise AI Platform. Technology: Developed component: OCR (Tesseract or PaddleOCR), Presidio analyzers, face and license plate detection, STT for audio, frame sampling for video. French version: ../../fr/bricks/F-11-multimodal-dlp.md.

Role and responsibilities

• Find personal and sensitive data in images (OCR text, faces, IDs), audio (spoken PII through transcripts) and video (sampled frames + audio).
• Apply modes detect, mask (blur boxes, bleep audio), tokenize (text), block, aligned with POST /v1/dlp/redact of brick E.

Position and interactions

• Execution path: synchronous for images, job for audio and video, inline in media upload.
• Invoked by policy obligations returned by POST /v1/policies/evaluate before any model call on media.
• Findings update the media classification and are written to the audit chain.

Open source and developed parts

• Open source reused: Presidio, Tesseract/PaddleOCR, OpenCV detectors, faster-whisper.
• Developed in the platform: Whole component: orchestration, bounding box masking, audio bleeping with timestamps, French recognizers (NIR, SIRET, IBAN FR).

Data model

Control Plane tables (schema eai): eai.media, eai.multimodal_jobs, eai.audit_events.

Every tenant table has tenant_id uuid NOT NULL and a row level security policy on current_setting('app.tenant_id', true)::uuid. ClickHouse usage_events is the analytics replica of eai.usage_events (ReplacingMergeTree keyed by event_id).

Contract: contracts/v1/multimodal-contract.schema.json (schema_version 1.0). Usage is always reported with contracts/v1/usage-event.schema.json, audit with contracts/v1/audit-event.schema.json.

API endpoints

Method	Path
POST	/v1/multimodal/dlp
POST	/v1/privacy/classify
POST	/v1/media

POST /v1/multimodal/dlp

Request:

{
  "media_id": "med_01HZX3K9",
  "mode": "mask"
}

Response:

{
  "job_id": "mmj_P01",
  "status": "succeeded",
  "findings": [
    {
      "type": "iban",
      "location": {
        "page": 1,
        "bbox": [
          412,
          1880,
          980,
          1920
        ]
      },
      "action": "masked"
    }
  ],
  "redacted_media_id": "med_RED1",
  "classification": "confidential"
}

POST /v1/privacy/classify

Request:

{
  "media_id": "med_01HZX3K9"
}

Response:

{
  "classification": "confidential",
  "findings": [
    {
      "type": "person_name",
      "count": 3
    }
  ]
}

POST /v1/media

Request:

{
  "filename": "invoice-scan.png",
  "mime_type": "image/png",
  "operation": "vision_understanding",
  "classification": "internal",
  "content_base64": "iVBORw0KGgo..."
}

Response:

{
  "media_id": "med_01HZX3K9",
  "media_type": "image",
  "mime_type": "image/png",
  "size_bytes": 482113,
  "resolution": "1654x2339",
  "classification": "internal",
  "dlp": {
    "status": "clean",
    "findings": 0
  },
  "storage": {
    "uri": "s3://eai-media/tenant/med_01HZX3K9",
    "encrypted": true
  }
}

SDK examples

Python

from enterprise_ai import EnterpriseAI

client = EnterpriseAI(base_url="<control-plane-url>", api_key="eai_live_xxx", tenant="acme", locale="en")
client.multimodal.audio.dlp("med_01HZX3K9", mode="mask")
client.privacy.classify(media_id="med_01HZX3K9")
client.multimodal.upload(open("invoice-scan.png", "rb").read(), "invoice-scan.png", "image/png", classification="internal")

TypeScript

import { EnterpriseAI } from "@enterprise-ai/sdk";

const client = new EnterpriseAI({ baseUrl: "<control-plane-url>", apiKey: process.env.EAI_API_KEY, tenant: "acme", locale: "en" });
await client.multimodal.dlp({ media_id: "med_01HZX3K9", mode: "mask" });
await client.privacy.classify({ media_id: "med_01HZX3K9" });
await client.media.upload({ file: new Blob([await readFile("invoice-scan.png")], { type: "image/png" }), filename: "invoice-scan.png", operation: "vision_understanding", classification: "internal" });

Java

EnterpriseAIClient client = EnterpriseAIClient.builder()
    .baseUrl("<control-plane-url>")
    .apiKey(System.getenv("EAI_API_KEY"))
    .tenant("acme")
    .locale(Locale.ENGLISH)
    .build();
client.multimodal().dlp(Map.of("media_id", "med_01HZX3K9", "mode", "mask"));
client.privacy().classify(Map.of("media_id", "med_01HZX3K9"));
client.media().upload(Map.of("content_base64", Base64.getEncoder().encodeToString(Files.readAllBytes(Path.of("invoice-scan.png"))), "mime_type", "image/png", "filename", "invoice-scan.png", "classification", "internal"));

Go

client, err := eai.NewClient(eai.WithBaseURL("<control-plane-url>"), eai.WithAPIKey(os.Getenv("EAI_API_KEY")), eai.WithTenant("acme"), eai.WithLocale("en"))
if err != nil { log.Fatal(err) }
ctx := context.Background()
_, err = client.Multimodal.Dlp(ctx, map[string]any{"media_id": "med_01HZX3K9", "mode": "mask"})
_, err = client.Privacy.Classify(ctx, map[string]any{"media_id": "med_01HZX3K9"})
_, err = client.Media.Upload(ctx, map[string]any{"content_base64": base64.StdEncoding.EncodeToString(scan), "mime_type": "image/png", "filename": "invoice-scan.png", "classification": "internal"})

// worker package: run a media worker consuming multimodal jobs
w, err := worker.New(worker.Config{Source: jobs, Handler: handle, Client: client, Concurrency: 2})
if err != nil { log.Fatal(err) }
err = w.Run(ctx)

Configuration

Environment variables: EAI_OCR_LANG.

Helm values keys (deploy/helm/enterprise-ai/values.yaml): workers.items[name=dlp].replicas.

Security

• Redacted media is a new media; the original stays encrypted with restricted access (security_officer).
• Findings never contain the raw value, only type, location and hash.
• Tenant resolved from the credential, never from the body; PostgreSQL row level security on every tenant table.
• Every mutation and policy decision writes a hash chained audit event (G-11).
• Errors use the standard envelope with message_i18n en and fr.

Observability

• Findings per type and per tenant, false positive feedback loop through evaluation suites.
• W3C traceparent propagated, X-Request-Id returned, spans carry eai.tenant_id, eai.project, eai.agent_id, eai.run_id.

FinOps units

• Recorded as cpu_seconds/gpu_seconds with labels.operation=dlp, documents or images counted.

Failure modes and resilience

• OCR failure -> classification raised to the safest level (fail closed), status needs_review.

Standalone adoption

A regulated client can adopt only multimodal DLP in front of its own AI stack.

Acceptance tests

1. A scanned RIB has its IBAN masked in the redacted media.
2. A phone number spoken at 00:42 is bleeped in the output audio.
3. An unreadable scan is classified confidential with needs_review.

Implementation roadmap

Kit L4 voice evidence

Realtime transcripts: DLP mask, tokenize or block before storage, client, model and logs; detect refused; stored rows, logs, audit and spans checked free of raw PII (MULT-009). See the voice runtime guide.