E-05 Document Processing

Brick E: Data, RAG, context, memory. French version: E-05-document-processing

1. Role and responsibilities

Converts PDF, Office, images and scans into structured text (headings, tables) with Docling and PaddleOCR.

• Structure and table extraction
• OCR of scanned pages
• Chunking for RAG

2. Position in the architecture and interactions

• Callers: Knowledge base ingestion, Connector Hub.
• Execution path: Asynchronous (job).
• Every interaction goes through the Control Plane API v1 (/v1) behind the API Gateway (D-14).

3. Open source component and developed part

• Open source: Docling, PaddleOCR
• Developed: Orchestration, chunking, ingestion classification.
• External integration through an adapter eai/integrations/<system>.py with an in-process fallback when the URL is not configured.

4. Data model

Tables (schema eai, tenant_id RLS):

• eai.kb_documents
• eai.kb_chunks
• eai.media

Common columns: id uuid, tenant_id uuid NOT NULL, created_at, updated_at; RLS policy current_setting('app.tenant_id', true)::uuid.

5. Contract

• Applied contract: multimodal-contract
• JSON Schema: contracts/v1/multimodal-contract.schema.json
• Key fields: document mime, pages, ocr
• Validation: POST /v1/contracts/{name}/validate; manifest apply: POST /v1/manifests/apply.

6. API endpoints

• POST /v1/knowledge-bases/{id}/documents
• GET /v1/knowledge-bases/{id}/documents

Request POST /v1/knowledge-bases/{id}/documents:

{
  "media_id": "med_44",
  "chunking": {
    "strategy": "structure",
    "max_tokens": 512,
    "overlap": 64
  },
  "ocr": "auto"
}

Response:

{
  "doc_id": "doc_210",
  "status": "processing"
}

7. SDK examples

Python:

from enterprise_ai import EnterpriseAI

client = EnterpriseAI(base_url="<control-plane-url>", api_key="eai_xxx_secret")
client.rag.ingest("kb_support", content_base64=pdf_base64, mime_type="application/pdf", title="contract.pdf")

TypeScript:

import { EnterpriseAI } from "@enterprise-ai/sdk";

const client = new EnterpriseAI({ baseUrl: "<control-plane-url>", apiKey: process.env.EAI_API_KEY });
await client.knowledgeBases.addDocuments("kb_support", { title: "Refund policy", content: "Refunds are paid within 14 days.", mime_type: "text/plain" });

Java:

EnterpriseAIClient client = EnterpriseAIClient.builder()
    .baseUrl("<control-plane-url>")
    .apiKey(System.getenv("EAI_API_KEY"))
    .build();
client.knowledge().addDocument("kb_support", Map.of("title", "Refund policy", "content", "Refunds are processed within 14 days."));

Go:

c, _ := eai.NewClient(eai.WithBaseURL("<control-plane-url>"), eai.WithAPIKey(os.Getenv("EAI_API_KEY")))
c.Knowledge.AddDocument(ctx, "kb_support", map[string]any{"uri": "https://<intranet-host>/refunds.pdf", "title": "Refund policy"})

8. Configuration

• Environment variables: EAI_DOCLING_URL, EAI_OCR_LANG, EAI_DATABASE_URL
• Helm values keys: controlPlane.config, workers.items

9. Security controls

• Antivirus scan before processing (Media F)
• Classification inherited by each chunk
• Every request carries the tenant resolved from the credential; PostgreSQL RLS
• Every mutation and policy decision writes a hash-chained audit event

10. Observability

• Prometheus metrics: eai_documents_processed_total{status}, eai_ocr_pages_total
• Traces: OpenTelemetry spans with GenAI semantic conventions (gen_ai.operation.name, gen_ai.agent.id, gen_ai.tool.name), attributes eai.tenant_id, eai.run_id, traceparent propagation.
• Audit actions: knowledge.document.ingest

11. FinOps units

• Processed pages, OCR pages, CPU/GPU.
• Emitted as Usage Event (POST /v1/finops/usage) with tenant, project, agent and run.

12. Failure modes and resilience

• Job retried by doc_id, idempotent by file sha256
• Normalized errors (error envelope, message_i18n en/fr)

13. Standalone adoption (client of a client)

• enterprise_ai.rag + Docling.
• SDK: enterprise_ai.privacy / dlp / knowledge_bases / memory / retention / lineage
• Minimal deployment: Helm chart with controlPlane.enabled=true and PostgreSQL; disabled external components use their in-process fallback.

14. Acceptance tests

1. Scanned PDF -> non empty OCR text
2. Cross-tenant access refused
3. Error returned in fr with Accept-Language: fr

Governed RAG (kit L4)

Docling Serve (EAI_DOCLING_URL) and PaddleOCR (EAI_PADDLEOCR_URL, eai.integrations.ocr, POST /ocr) go through eai.security.egress. Without a service the builtin extraction (text, HTML, PDF) and tesseract stay active.

Verified Docling and PaddleOCR toolchains (DATA-010, DATA-011)

Setup (separate virtual environments, pinned versions, official sources only):

bash tools/rag/setup-docling.sh     # /tmp/toolchains/docling-venv + /tmp/toolchains/docling-models
bash tools/rag/setup-paddleocr.sh   # /tmp/toolchains/paddleocr-venv + /tmp/toolchains/paddleocr-models

• Docling 2.145.0 (MIT), docling-core 2.105.0 (MIT), docling-ibm-models 4.0.4 (MIT), torch 2.9.1+cpu (BSD-3-Clause, official index https://download.pytorch.org/whl/cpu), python-docx 1.2.0 (MIT).
• paddlepaddle 3.3.1 CPU (Apache-2.0), paddleocr 3.7.0 (Apache-2.0), paddlex 3.7.2 (Apache-2.0), from PyPI.
• The scripts write models.sha256 (sha256 of every weight file) and print the licence of every model card.

Model weights and licences:

Model	Official source	File	sha256	Licence
docling-layout-heron	Hugging Face docling-project/docling-layout-heron	model.safetensors	00333a43451945aaf89db8ca9c0a17e75d1537c17db60fdb91aa95f4c7929e0c	Apache-2.0
docling-layout-heron-onnx	Hugging Face docling-project/docling-layout-heron-onnx	model.onnx	59c81a3a2923042d85034ffc487f8f47e4854117e879aef89b2b9f728fb4922a	Apache-2.0
TableFormer accurate	Hugging Face docling-project/docling-models	tableformer_accurate.safetensors	2a7d6c924b3cd12fb99a09280ca9c33a89c5d60b93253617d2e088c1a40374d9	CDLA-Permissive-2.0
TableFormer fast	Hugging Face docling-project/docling-models	tableformer_fast.safetensors	3119563aab5a7c96fda4d621119b63fd8806272b86c30936d15507616422f718	CDLA-Permissive-2.0
PP-OCRv5_mobile_det	Hugging Face PaddlePaddle/PP-OCRv5_mobile_det	inference.pdiparams	afa1820cb16c1fd0dad589d0f8b389139061c1ef6d68019685fd07be997dda5b	Apache-2.0
PP-OCRv5_mobile_rec	Hugging Face PaddlePaddle/PP-OCRv5_mobile_rec	inference.pdiparams	2460da90875937c94db97eba74ae3d9e5d4c4c57c42f1f41531c09a26bcc771a	Apache-2.0

HTTP workers (127.0.0.1 only, free port chosen by the caller):

HF_HUB_OFFLINE=1 /tmp/toolchains/docling-venv/bin/python tools/rag/docling_worker.py --port PORT --models /tmp/toolchains/docling-models
PADDLE_PDX_CACHE_HOME=/tmp/toolchains/paddleocr-models /tmp/toolchains/paddleocr-venv/bin/python tools/rag/paddleocr_worker.py --port PORT

docling_worker.py implements the POST /v1/convert/source subset of Docling Serve 1.x (sources of kind file, response document.md_content), with Docling OCR off (scans go to PaddleOCR, ocr_applied: false is returned). paddleocr_worker.py implements POST /ocr of PaddleX serving; oneDNN is off (enable_mkldnn=False) because paddlepaddle 3.3.1 CPU raises NotImplementedError (ConvertPirAttribute2RuntimeAttribute) on these models. The adapter adds the extension of the MIME type to the file name sent to Docling, which picks the input format from the extension.

Evidence: services/control-plane/tests/test_rag_docling_ocr_real.py starts both workers and checks, through eai.integrations.docling and eai.integrations.ocr (hence eai.security.egress), an acceptance PDF (text and table), a DOCX with a table (exact cells), ingestion through the API then a query, the refusal of a destination outside the egress policy, and OCR of an image of text rendered with Pillow. Missing toolchain: explicit failure, never skipped.