Skip to main content

E-07 Vector Database

Brick E: Data, RAG, context, memory. French version: E-07-vector-database

1. Role and responsibilities​

Stores chunk vectors in Qdrant with payload (tenant_id, kb_id, acl, classification).

  • One collection per knowledge base or per tenant
  • Mandatory tenant_id filter
  • Snapshots and replication

2. Position in the architecture and interactions​

  • Callers: RAG Pipelines (E-06), Memory Manager (E-10).
  • Execution path: Synchronous (gRPC/HTTP).
  • Every interaction goes through the Control Plane API v1 (/v1) behind the API Gateway (D-14).

3. Open source component and developed part​

  • Open source: Qdrant
  • Developed: Adapter eai/integrations/qdrant.py and in-process fallback.
  • External integration through an adapter eai/integrations/<system>.py with an in-process fallback when the URL is not configured.

4. Data model​

Tables (schema eai, tenant_id RLS):

  • eai.kb_chunks (vector_id)
  • eai.knowledge_bases (collection)

Common columns: id uuid, tenant_id uuid NOT NULL, created_at, updated_at; RLS policy current_setting('app.tenant_id', true)::uuid.

5. Contract​

  • Applied contract: memory-contract
  • JSON Schema: contracts/v1/memory-contract.schema.json
  • Key fields: embedding_ref
  • Validation: POST /v1/contracts/{name}/validate; manifest apply: POST /v1/manifests/apply.

6. API endpoints​

  • POST /v1/knowledge-bases/{id}/query
  • POST /v1/memory/search

Request POST /v1/memory/search:

{
  "query": "préférences de langue",
  "scope": { "type": "user", "id": "usr_42" },
  "top_k": 3
}

Response:

{
  "items": [
    {
      "memory_id": "mem_5",
      "score": 0.88,
      "content": "Préfère le français"
    }
  ]
}

7. SDK examples​

Python:

from enterprise_ai import EnterpriseAI

client = EnterpriseAI(base_url="<control-plane-url>", api_key="eai_xxx_secret")
client.memory.search(query="préférences", scope={"type": "user", "id": "usr_42"}, top_k=3)

TypeScript:

import { EnterpriseAI } from "@enterprise-ai/sdk";

const client = new EnterpriseAI({ baseUrl: "<control-plane-url>", apiKey: process.env.EAI_API_KEY });
await client.memory.search({ query: "preferences", scope: { type: "user", id: "usr_42" }, top_k: 3 });

Java:

EnterpriseAIClient client = EnterpriseAIClient.builder()
    .baseUrl("<control-plane-url>")
    .apiKey(System.getenv("EAI_API_KEY"))
    .build();
client.memory().search(Map.of("query", "preferences", "scope", Map.of("type", "user", "id", "usr_42"), "top_k", 3));

Go:

c, _ := eai.NewClient(eai.WithBaseURL("<control-plane-url>"), eai.WithAPIKey(os.Getenv("EAI_API_KEY")))
c.Memory.Search(ctx, map[string]any{"query": "preferences", "scope": map[string]any{"type": "user", "id": "usr_42"}, "top_k": 3})

8. Configuration​

  • Environment variables: EAI_QDRANT_URL, EAI_QDRANT_API_KEY, EAI_DATABASE_URL
  • Helm values keys: qdrant.enabled, qdrant.replicaCount, qdrant.persistence.size

9. Security controls​

  • Qdrant not exposed outside the cluster
  • Tenant filter added server side
  • Every request carries the tenant resolved from the credential; PostgreSQL RLS
  • Every mutation and policy decision writes a hash-chained audit event

10. Observability​

  • Prometheus metrics: qdrant_collections_vector_total, eai_vector_search_latency_seconds
  • Traces: OpenTelemetry spans with GenAI semantic conventions (gen_ai.operation.name, gen_ai.agent.id, gen_ai.tool.name), attributes eai.tenant_id, eai.run_id, traceparent propagation.
  • Audit actions: knowledge.query, memory.search

11. FinOps units​

  • Vector storage (GB-month), queries.
  • Emitted as Usage Event (POST /v1/finops/usage) with tenant, project, agent and run.

12. Failure modes and resilience​

  • Replication factor 2, snapshots to S3 (E-13)
  • Reindex possible from eai.kb_chunks
  • Normalized errors (error envelope, message_i18n en/fr)

13. Standalone adoption (client of a client)​

  • Qdrant + Control Plane (qdrant.enabled=true).
  • SDK: enterprise_ai.privacy / dlp / knowledge_bases / memory / retention / lineage
  • Minimal deployment: Helm chart with controlPlane.enabled=true and PostgreSQL; disabled external components use their in-process fallback.

14. Acceptance tests​

  1. Tenant A query never returns a tenant B vector
  2. Cross-tenant access refused
  3. Error returned in fr with Accept-Language: fr

Governed RAG (kit L4)​

Real Qdrant (official 1.20.0 binary, tools/rag/fetch.sh, sha256 verified): one collection per tenant and embedding space, server side filters kb_id, readers, chunk_readers, classification_level and active. tests/test_rag_qdrant_real.py proves no point of tenant B is returned to tenant A.