Load testing plan
k6 scripts in deploy/loadtest/. Run on staging only.
Measured local results (throughput, latencies, quotas, RTO and RPO): performance-results.md.
| Script | Goal | Thresholds |
|---|---|---|
chat-completions.js | Gateway /v1/gateway/chat/completions, steady 20 req/s then ramp to 150 req/s | errors < 1 %, p95 < 8 s, TTFB p95 < 2 s |
control-plane-api.js | Management API (reads, audit), 300 VUs | errors < 0.5 %, p95 < 300 ms, p99 < 1 s |
rag-agents.js | RAG queries and agent runs | RAG p95 < 800 ms |
k6 run -e BASE_URL=<control-plane-url> -e EAI_API_KEY=$KEY deploy/loadtest/chat-completions.js
Watch: HPA and KEDA ScaledObjects (vLLM vllm:num_requests_waiting, JetStream lag), PostgreSQL pool, OPA/OpenFGA latency, the Live metrics and GPU allocation portal screens, the vllm:* and DCGM_FI_* metrics in Prometheus. Exit criteria: no threshold breached, no saturation 5xx, vLLM scale up time recorded.