Measured performance, quotas, RTO and RPO
Requirements: OBSE-013, OBSE-014, LLM_-009, RELE-014, OBSE-010. Version française : ../performance-results.md.
Every figure on this page comes from a run of bash tools/perf/run.sh on the host described below, on 2026-10-30 (results folder /tmp/eai-perf-final). Nothing is estimated. The figures depend on the host (WSL2 shared with other processes): they are a local capacity reference, not a service commitment.
Host
nproc: 6
uname -sr: Linux 6.6.87.2-microsoft-standard-WSL2
free -g:
total used free shared buff/cache available
Mem: 47 20 3 0 23 26
Swap: 96 1 94
Commands
bash tools/perf/run.sh /tmp/eai-perf-final
cd services/control-plane && .venv/bin/python -m pytest -q tests/test_perf_measurements.py tests/test_finops_attribution.py tests/test_lineage_exploration.py
tools/perf/loadgen.py (Python asyncio and httpx, no other tool) starts on free ports a throwaway PostgreSQL (pgserver of services/control-plane/.venv, every migration applied), the fake OpenAI compatible provider tools/perf/fake_upstream.py and the Control Plane under uvicorn (one process, EAI_AUTH_DEV=1, EAI_EGRESS_PRIVATE_ALLOWLIST=127.0.0.1/32, global rate limiting off so that the platform is measured and not the limiter). CPU time (utime + stime) and RSS of the API process are read from /proc/<pid>/stat and /proc/<pid>/status. Results go to load.json and load.md. tools/perf/resilience.py writes resilience.json.
The k6 scripts of deploy/loadtest were not run: the k6 binary is AGPL-3.0 and may only be used as a local test tool, never distributed; the in-house generator is used instead.
Throughput and latency (OBSE-014)
400 requests per concurrency level after 4 warm-up requests. Scenarios: health (GET /healthz), model_gateway_chat (POST /v1/gateway/chat/completions to the fake provider: quota reservation, PEP, DLP, call, Usage Event, commit), tool_gateway_invoke (POST /v1/tools/{name}/invoke on the builtin echo handler: the egress guard refuses private destinations for tenant tools by design, so the measured path is the full tool gateway without a network hop). The CPU column is the CPU time used by the API during the level; RSS is read at the end of the level.
| Scenario | Concurrency | Requests | Errors | Error rate | Throughput (req/s) | p50 (ms) | p95 (ms) | p99 (ms) | API CPU (s) | API RSS (MiB) |
|---|---|---|---|---|---|---|---|---|---|---|
| health | 1 | 400 | 0 | 0.000 | 368.13 | 2.6 | 3.37 | 4.47 | 0.77 | 127.7 |
| health | 8 | 400 | 0 | 0.000 | 502.51 | 14.57 | 19.93 | 82.08 | 0.74 | 128.3 |
| health | 32 | 400 | 0 | 0.000 | 421.47 | 54.79 | 213.32 | 299.12 | 0.73 | 130.2 |
| model_gateway_chat | 1 | 400 | 0 | 0.000 | 13.09 | 73.92 | 91.92 | 110.68 | 22.43 | 209.7 |
| model_gateway_chat | 8 | 400 | 0 | 0.000 | 19.9 | 397.87 | 523.22 | 565.42 | 34.52 | 222.3 |
| model_gateway_chat | 32 | 400 | 0 | 0.000 | 20.4 | 1548.02 | 1960.26 | 2147.06 | 50.18 | 261.3 |
| tool_gateway_invoke | 1 | 400 | 0 | 0.000 | 34.56 | 25.65 | 46.71 | 66.93 | 6.16 | 261.2 |
| tool_gateway_invoke | 8 | 400 | 0 | 0.000 | 46.96 | 163.84 | 244.77 | 302.09 | 7.61 | 261.2 |
| tool_gateway_invoke | 32 | 400 | 0 | 0.000 | 42.2 | 727.72 | 938.55 | 1039.71 | 9.25 | 261.5 |
Reading: one uvicorn process tops out around 20 model gateway calls per second on this host; beyond that, concurrency only adds waiting (latency grows linearly). Scaling goes through several replicas, not measured here (see blocked items).
Quotas (LLM_-009)
| Limit | Configuration | Observed answers | Error code | Recovery |
|---|---|---|---|---|
| Requests per minute | tool manifest rate_limit.requests_per_minute: 5 | 5 x 200 then 3 x 429, Retry-After: 60 | rate_limited | 200 again 60.6 s after the first refusal (60 s sliding window) |
| Tokens | POST /v1/quotas tenant, period: hour, limit_tokens: 100 | 4 x 200 (96 tokens used) then 429 before any provider call | quota_exceeded | 200 as soon as the limit is raised with PUT /v1/quotas/{id} |
| Concurrent reservations | POST /v1/quotas tenant, period: day, limit_cost: 0.30, reservations of 0.10 through POST /v1/quotas/reserve | 3 x 201 then 2 x 429 | quota_exceeded | 201 as soon as they are released (POST /v1/quotas/{id}/release) |
The quota service has no period shorter than one hour (request, run, hour, day, month): a tokens per minute limit does not exist as such, the finest measured token limit is the hourly one. Recovery of the hourly limit after its window (one hour) was not waited for; recovery was measured by raising the limit.
Cost attribution (OBSE-013) and lineage (OBSE-010)
Tenant ids are deliberately not Prometheus labels (cardinality and isolation). Attribution per tenant, model and agent is proven by services/control-plane/tests/test_finops_attribution.py through GET /v1/finops/usage, GET /v1/finops/summary?group_by=model|agent|tenant and the analytics export rows sent to ClickHouse, with no cross-tenant leakage. The lineage walk from answer to source and back is proven by services/control-plane/tests/test_lineage_exploration.py.
RTO and RPO (RELE-014)
RTO and RPO targets: to be set by the client. Only measured values are shown.
| Component | Measure | Measured value |
|---|---|---|
| PostgreSQL | database size (migrations + 50,090 probe rows) | 18,706,915 bytes |
| PostgreSQL | backup scripts/pg_backup.sh (custom format, checked with pg_restore --list) | 0.16 s, 816,324 bytes |
| PostgreSQL | restore scripts/pg_restore.sh into an empty database, with the migration check | 9.059 s |
| PostgreSQL | data loss of a logical backup (writes every 10 ms) | 50 rows, 0.636 s window written after the snapshot |
| Redis | replica promotion (REPLICAOF NO ONE) after SIGKILL of the primary, up to the first accepted write | 4.071 s |
| Redis | keys acknowledged by the primary and missing on the promoted replica | 0 of 350 |
The loss of a logical backup is everything written after the snapshot: the real RPO of a restore is the interval between two backups plus this window. The Redis promotion time runs from the SIGKILL to the first write accepted by the promoted replica; failure detection by Sentinel or the operator is not included. A first trial the same day gave 2.907 s of restore and 0.0021 s of promotion: the variance of the shared host is high, these values are not bounds.
Blocked items
- GPU measurements: BLOCKED, no GPU on this host.
- High availability and scale in a cluster (several replicas, CloudNativePG PostgreSQL, Redis Sentinel): BLOCKED, no cluster available; nothing was applied to a cluster or a cloud account.