Skip to main content

H-04 GPU accounting

Brick H of the Enterprise AI Platform. Technology: NVIDIA DCGM exporter + Prometheus, sampler pushing to /v1/finops/gpu/samples, allocation policies. French version: ../../fr/bricks/H-04-gpu-accounting.md.

Role and responsibilities​

  • Measure GPU-seconds, GPU memory, power (estimated energy) per pod, deployment and tenant.
  • Allocate the cost of local models (GPU, infra, energy, storage) to tenants according to real usage.

Position and interactions​

  • Execution path: batch (periodic sampling and allocation).
  • Shared vLLM servers are split by tokens served per tenant (gateway usage) weighted on GPU-seconds; dedicated deployments go fully to the owner.

Open source and developed parts​

  • Open source reused: DCGM exporter, Prometheus.
  • Developed in the platform: Sampler, allocation engine, idle capacity policy.

Data model​

Control Plane tables (schema eai): eai.gpu_samples, eai.allocation_policies, eai.usage_events.

Every tenant table has tenant_id uuid NOT NULL and a row level security policy on current_setting('app.tenant_id', true)::uuid. ClickHouse usage_events is the analytics replica of eai.usage_events (ReplacingMergeTree keyed by event_id).

Contract: contracts/v1/usage-event.schema.json (schema_version 1.0). Usage is always reported with contracts/v1/usage-event.schema.json, audit with contracts/v1/audit-event.schema.json.

API endpoints​

MethodPath
POST/v1/finops/gpu/samples
GET/v1/finops/gpu/allocation
POST/v1/finops/allocation-policies
GET/v1/providers/catalog

POST /v1/finops/gpu/samples​

Request:

{
  "samples": [
    {
      "node": "gpu-node-3",
      "gpu_uuid": "GPU-5e1f",
      "model": "H100-80GB",
      "namespace": "eai-inference",
      "pod": "vllm-llama-70b-0",
      "tenant_id": "8f0c2a4e-1b7d-4c55-9a51-2f3e6d7c9b10",
      "deployment_id": "vllm-llama-70b",
      "window_seconds": 15,
      "utilization": 0.82,
      "memory_used_gb": 61.2,
      "power_w": 512
    }
  ]
}

Response:

{
  "accepted": 1
}

GET /v1/finops/gpu/allocation​

Response:

{
  "period": "2026-10",
  "items": [
    {
      "tenant_id": "8f0c2a4e-1b7d-4c55-9a51-2f3e6d7c9b10",
      "deployment_id": "vllm-llama-70b",
      "gpu_seconds": 182300,
      "gpu_memory_gb_seconds": 10230000,
      "energy_wh": 25900,
      "allocated": {
        "amount": "412.70",
        "currency": "EUR"
      },
      "policy_id": "alp_1"
    }
  ]
}

POST /v1/finops/allocation-policies​

Request:

{
  "name": "local-gpu-default",
  "deployment_pattern": "*",
  "currency": "EUR",
  "gpu_hourly_cost": "2.10",
  "overhead_pct": "18",
  "energy_kwh_price": "0.21",
  "pue": "1.4",
  "storage_gb_month_price": "0.02"
}

Response:

{
  "id": "alp_1",
  "status": "active"
}

GET /v1/providers/catalog​

Response:

{
  "items": [
    {
      "id": "vllm",
      "label": "vLLM (local inference engine)",
      "local": true,
      "base_url": "http://vllm-llama-70b:8000/v1",
      "provider_id": "7fcf4489-04d6-4992-9187-0ccca12986e6",
      "enabled": true,
      "discovery": "api"
    }
  ]
}

SDK examples​

Python​

from enterprise_ai import EnterpriseAI

client = EnterpriseAI(base_url="<control-plane-url>", api_key="eai_live_xxx", tenant="acme", locale="en")
client.finops.gpu_samples(samples=samples)
client.finops.gpu_allocation(start="2026-10-01", end="2026-10-31")
client.finops.create_allocation_policy({"name": "local-gpu-default", "deployment_pattern": "*"})
client.providers.catalog()

TypeScript​

import { EnterpriseAI } from "@enterprise-ai/sdk";

const client = new EnterpriseAI({ baseUrl: "<control-plane-url>", apiKey: process.env.EAI_API_KEY, tenant: "acme", locale: "en" });
await client.finops.gpu.ingestSamples({ samples: samples });
await client.finops.gpu.allocation({ from: "2026-10-01", to: "2026-10-31" });
await client.finops.allocationPolicies.create({ name: "local-gpu-default", gpu_hourly_cost: "2.50", currency: "EUR" });
await client.providers.catalog();

Java​

EnterpriseAIClient client = EnterpriseAIClient.builder()
    .baseUrl("<control-plane-url>")
    .apiKey(System.getenv("EAI_API_KEY"))
    .tenant("acme")
    .locale(Locale.ENGLISH)
    .build();
client.finops().gpuSamples(Map.of("samples", samples));
client.finops().gpuAllocation(Map.of("from", "2026-10-01", "to", "2026-10-31"));
client.finops().createAllocationPolicy(Map.of("name", "local-gpu-default", "gpu_model", "L40S", "gpu_hourly_cost", 2.10));
client.providers().catalog();

Go​

client, err := eai.NewClient(eai.WithBaseURL("<control-plane-url>"), eai.WithAPIKey(os.Getenv("EAI_API_KEY")), eai.WithTenant("acme"), eai.WithLocale("en"))
if err != nil { log.Fatal(err) }
ctx := context.Background()
_, err = client.FinOps.GpuSamples(ctx, map[string]any{"samples": samples})
_, err = client.FinOps.GpuAllocation(ctx, eai.WithQuery(map[string]string{"from": "2026-10-01", "to": "2026-10-31"}))
_, err = client.FinOps.CreateAllocationPolicy(ctx, map[string]any{"name": "local-gpu-default", "gpu_hourly_cost": "2.50"})
_, err = client.Providers.Catalog(ctx)

// usage package: emit a Usage Event from a local worker
emitter := usage.NewEmitter(client, usage.Options{})
defer emitter.Close(ctx)
ev, err := usage.NewEvent("acme", "gpu_local", "worker").Model("local", "llama-70b-local", "", "").Unit("gpu_seconds", 12.4).Unit("energy_wh", 1.7).Build()
if err == nil { err = emitter.Emit(ev) }
// gatewaymw: reserve, call, commit around any http.Handler
http.Handle("/llm", gatewaymw.New(gatewaymw.Config{Client: client, Action: func(r *http.Request) string { return "llm.invoke" }, Reserve: func(r *http.Request) map[string]any { return map[string]any{"cost": "0.50", "tokens": 4000} }})(llmHandler))

Configuration​

Environment variables: EAI_DATABASE_URL, EAI_CLICKHOUSE_URL.

Helm values keys (deploy/helm/enterprise-ai/values.yaml): vllm.nodeSelector, global.monitoring.

Security​

  • Samples carry no workload content; pushing samples needs a service identity.
  • Tenant resolved from the credential, never from the body; PostgreSQL row level security on every tenant table.
  • Every mutation and policy decision writes a hash chained audit event (G-11).
  • Errors use the standard envelope with message_i18n en and fr.

Observability​

  • GPU utilization, memory, power and idle ratio dashboards.
  • W3C traceparent propagated, X-Request-Id returned, spans carry eai.tenant_id, eai.project, eai.agent_id, eai.run_id.

FinOps units​

  • Units gpu_seconds, gpu_memory_gb_seconds, energy_wh; allocated cost recorded as usage_type gpu_local with allocation_policy_id.

Failure modes and resilience​

  • Missing samples window -> interpolation flagged estimated, never silently zero.

Standalone adoption​

On-prem clients use it alone for GPU chargeback.

Acceptance tests​

  1. Two tenants sharing a deployment 70/30 in tokens get 70/30 of the allocated cost.

Implementation roadmap