Skip to main content

Live metrics

The Live metrics screen lets you watch, without refreshing, the requests, errors and p95 latency of the last minute, the new traces and the incident changes of your tenant, pushed by the server (Server-Sent Events). Portal path: /observability/metrics (menu Observability > Live metrics). Support engineers and administrators keep it open during a deployment or an incident.

What the screen shows​

AreaContent
Connection badgeLive while the stream is open, Reconnecting while the portal opens it again.
CardsRequests (1 min), Errors (1 min), Error rate, p95 latency (ms) of the last minute.
Live chart tabRequests, errors and p95 latency: one point per tick of the server stream (last minute window). Waiting for the first measurement until the first tick.
New traces tabRoot spans received since the page opened: Start, Name, Status, Duration (ms), Trace id. The tab shows their count.
Incident changes tabIncidents created or changed since the page opened: Updated, Title, Severity, Status, Change (created or updated). The tab shows their count.

The tab is kept in the address (?tab=traces, ?tab=incidents).

Actions​

Button or actionWhat it doesAPI callRoles
Opening the screenOpens the stream for 300 seconds; the server sends a metrics event at each tick (every 2 seconds by default), a trace event per new root span and an incident event per incident change.GET /v1/telemetry/stream (timeout=300)platform_admin, tenant_admin, auditor, viewer, support_engineer
Automatic reconnectionWhen the stream closes (timeout or network loss), the portal waits 3 seconds and reopens it with the Last-Event-ID header, so no event is lost.GET /v1/telemetry/streamSame roles
Live chart, New traces, Incident changesSwitch the tab.NoneSame roles

The screen has no button that changes data.

Roles​

RoleAccess
platform_admin, tenant_admin, auditor, viewer, support_engineerOpen the screen and follow the stream of their tenant (ROLES of services/control-plane/src/eai/domains/observability/slo.py, reused by services/control-plane/src/eai/domains/observability/live.py).
Other rolesThe Observability section is hidden; the page shows Your roles do not open this page.

The stream needs a tenant context: the rows are those of the tenant of the caller.

Typical tasks​

Watch a deployment​

  1. Open the screen. Go to Observability > Live metrics before the deployment starts. Expected result: the badge shows Live and the cards fill within a few seconds. If it fails: if the badge stays on Reconnecting, check the network path to the Control Plane (a proxy must not buffer text/event-stream).
  2. Follow the chart. In Live chart, watch the Errors (1 min) and p95 latency (ms) lines while the new version rolls out. Expected result: the error line stays flat and the latency stays under its target. If it fails: open New traces and click through to Traces to find the failing requests.

Follow an incident​

  1. Open the Incident changes tab. Every incident created or changed appears without refresh. Expected result: the tab count grows when an incident changes, with Change set to created or updated. If it fails: No incident created or changed since the page opened. means nothing changed yet.
  2. Open the incident. Go to Incidents to act on it. Expected result: the incident shows the same status. If it fails: check that you have a role of the security section.

Troubleshooting​

SymptomCauseFix
Badge stuck on ReconnectingThe stream cannot be opened: network, proxy buffering, or an error answer.Check the proxy (no buffering for text/event-stream) and your session.
Waiting for the first measurement staysNo tick received yet.Wait a few seconds; reload the page.
Cards at zeroNo root span in the last minute.Expected when the tenant has no traffic; traces arrive with POST /v1/telemetry/spans.
Error forbidden or tenant_required (HTTP 403) on the streamNo observability role, or no tenant.Ask for the viewer or support_engineer role in your tenant.