Skip to main content

Lineage

The Lineage screen lets you browse the OpenLineage graph of the platform: which job reads which dataset and produces which other one. Portal path: /data/lineage. Data stewards, auditors and developers use it to follow where the data of an agent or a knowledge base comes from.

What the screen shows​

The Graph section loads the graph once and draws it in layers: a node is placed one column to the right of the nodes it depends on (up to 13 columns). Datasets are drawn as rectangles and jobs (type job) as rounded shapes; each node shows the first 20 characters of its name, and arrows follow the edges.

A click on a node selects it and shows its record on the right: id, type, namespace, name, facets, last_event_type, updated_at. Before any click, the panel says "Select a node to see its details."

Without any event, the screen shows "No lineage recorded" and the hint "Ingestion jobs and agents emit OpenLineage events." The server returns at most 500 nodes and leaves out the run nodes.

Actions​

Button or actionWhat it doesAPI callRoles
Opening the screenLoads the nodes and the edgesGET /v1/lineage/graphany tenant user
Click on a nodeShows the record of the node (no API call)noneany tenant user
Retry (error state)Loads the graph againGET /v1/lineage/graphany tenant user

Roles​

The screen has no write action. The Control Plane (services/control-plane/src/eai/domains/lineage/__init__.py) only requires a tenant for the graph; the events are written by the ingestion jobs and the agents through POST /v1/lineage/events, not from this screen.

Typical tasks​

Find where a knowledge base gets its documents​

  1. Open the graph. Every ingestion emits a run that links its inputs and outputs. Open Lineage.

    Expected result: the layered graph with datasets and jobs.

    If it fails: "No lineage recorded" means no job has emitted an event yet; ingest a document in Knowledge bases.

  2. Select the dataset. Click the node of the dataset.

    Expected result: its record on the right, with its namespace and facets.

    If it fails: if the name is cut, read the full name in the record.

Follow one dataset upstream and downstream​

  1. Use the dataset view. The full graph can be large. Open RAG quarantine, tab Dataset lineage, type the dataset id, then Apply.

    Expected result: the producers, the consumers and the graphs on both sides of the dataset.

    If it fails: not_found means the id is not a known dataset.

Troubleshooting​

SymptomCauseFix
"No lineage recorded"No OpenLineage event was receivedRun an ingestion or an agent that emits lineage
Error state with RetryThe Control Plane did not answer the graph requestClick Retry, then check the API
tenant_required (403)Your session has no tenantSign in with an account of a tenant
A node has no arrowIts edges point to nodes beyond the 500 returnedUse the dataset view of RAG quarantine