Skip to main content

LLM projects

The LLM projects screen lets you create the LLM projects of the tenant, choose the models each one may use and set its quota (budget, tokens, RPM, TPM, parallel requests), mirrored into a LiteLLM team. Portal path: /llm/projects. It is used by the tenant and platform administrators (tenant_admin, platform_admin). The screen is built in portal/src/screens/llm-management.tsx (LlmProjects, ProjectDialog, QuotaDialog) with the helpers of portal/src/lib/llm-management.ts.

What the screen shows​

  • The header button New project (administrators).
  • The section LiteLLM admin API (GET /v1/llm/management/status), in one of three states:
    • ready: "LiteLLM answers at <url>: projects are mirrored as teams and keys are generated there.";
    • LiteLLM is not configured: projects are saved but not mirrored, and no key can be created; the missing settings are listed, with Check again;
    • LiteLLM does not answer at <url>: projects stay saved and can be synchronized later, with Check again.
  • The section LLM projects ("Each project is a LiteLLM team: its models and limits apply to every key of the project."), a table read from GET /v1/llm/projects:
ColumnContent
NameName of the LLM project.
Statusactive, suspended or archived.
LiteLLMSynchronization state of the team: synced, pending (LiteLLM not configured) or failed.
ModelsEffective models (allowed models plus the models of the allowed categories), or All models.
Budget<amount> <currency> / <period>, or no limit.
Rate limits<n> RPM, <n> TPM, no limit when empty.
KeysNumber of keys not revoked.

When there is no project: "No LLM project. Create a project, set its quota, then create its keys."

The project form (New project, Edit the project) has the fields Name, Owner (you by default), Description, Tenant project (optional link to a project of the tenant, read from GET /v1/projects: its daily and monthly limits are then enforced by the gateway), Status, Allowed models (suggestions from GET /v1/models) and Allowed categories. Leave both empty to allow every model. On creation the quota fields are added: Budget (blank: no budget), Currency (USD by default), Budget period (daily, weekly, monthly, total), Token limit, Token period, Requests per minute (RPM), Tokens per minute (TPM), Max parallel requests, Alert threshold (%) (80 by default, soft alert), Hard stop (on by default; off: the budget only raises alerts).

Actions​

Button or actionWhat it doesAPI callRoles
Check again (LiteLLM admin API)Probes the LiteLLM admin API again.GET /v1/llm/management/statustenant_admin, platform_admin
New project (header or empty state)Creates the project with its quota, then mirrors it into a LiteLLM team. Toast "Project saved and synchronized with LiteLLM" or "Project saved, but not synchronized with LiteLLM".POST /v1/llm/projectstenant_admin, platform_admin
Keys of the project (row menu)Opens LiteLLM keys on this project (/llm/keys?project=<id>).noneeveryone who sees the table
Edit the project (row menu)Changes name, owner, description, tenant project, status and allowed models; an empty Tenant project unlinks it.PATCH /v1/llm/projects/{project_id}tenant_admin, platform_admin
Quota (row menu)Dialog Quota of <name> with the quota fields; saves the quota, updates the Control Plane quotas and the LiteLLM team.PUT /v1/llm/projects/{project_id}/quotatenant_admin, platform_admin
Synchronize with LiteLLM (row menu)Pushes the project to LiteLLM again (team creation or update). Toast "Project synchronized with LiteLLM".POST /v1/llm/projects/{project_id}/synctenant_admin, platform_admin
Delete the project (row menu)Destructive confirmation listing the LiteLLM team, the risk "The LiteLLM team and the Control Plane quotas of the project are deleted." and the required roles; type the resource name, then Confirm.DELETE /v1/llm/projects/{project_id}tenant_admin, platform_admin

Roles​

  • Every /v1/llm/projects route and GET /v1/llm/management/status require ADMIN of services/control-plane/src/eai/domains/llm_management/__init__.py: tenant_admin or platform_admin (MANAGERS in portal/src/lib/llm-management.ts). Other roles, ai_platform_engineer included, get forbidden and see an error panel.
  • The buttons New project, Edit the project, Quota, Synchronize with LiteLLM and Delete the project are shown only to these two roles.
  • Every route needs a tenant: a principal without tenant gets tenant_required.

Typical tasks​

Create a project with a monthly budget​

  1. Check LiteLLM. Read the section LiteLLM admin API, because a project is mirrored into LiteLLM only when its admin API answers. Expected result: the ready message with the LiteLLM address. If it fails: set the settings listed (see the Troubleshooting section), then Check again.
  2. Open the form. Click New project. Expected result: the dialog with the project and quota fields. If it fails: the button is missing when you are neither tenant_admin nor platform_admin.
  3. Fill the project. Type the Name, pick the Allowed models or Allowed categories (both empty: every model), then Budget, Currency, Budget period monthly and the rate limits, so that every key of the project inherits them. Expected result: no validation message. If it fails: a required field is marked Required.
  4. Create. Click New project. Expected result: toast "Project saved and synchronized with LiteLLM" and LiteLLM synced in the table. If it fails: "Project saved, but not synchronized with LiteLLM" means the project exists with pending or failed; fix LiteLLM, then use Synchronize with LiteLLM.

Tighten the quota of a project​

  1. Open the quota. Row menu > Quota. Expected result: the dialog Quota of <name> with the current values. If it fails: refresh the page.
  2. Change the limits. Lower Budget, Requests per minute (RPM) or Tokens per minute (TPM), and keep Hard stop on if calls must stop at the limit. Expected result: the values are accepted. If it fails: "Invalid quota: <reason>." names the wrong value.
  3. Save. Click Save. Expected result: toast "Quota saved and synchronized with LiteLLM" and the new Budget and Rate limits in the table. If it fails: the quota is saved even when LiteLLM does not answer; synchronize later.

Delete a project​

  1. Revoke its keys. Open Keys of the project and revoke every key, because a project with active keys cannot be deleted. Expected result: Keys shows 0. If it fails: see LiteLLM keys.
  2. Delete. Row menu > Delete the project, type the resource name, then Confirm. Expected result: toast Project deleted; the LiteLLM team and the Control Plane quotas are removed. If it fails: "This project still has active keys: revoke them first."

Troubleshooting​

SymptomCauseFix
LiteLLM is not configured (litellm_not_configured, 503)The admin address or the master key reference is missing.Set EAI_LITELLM_ADMIN_URL (or EAI_LITELLM_URL) and EAI_LITELLM_MASTER_KEY_REF (or EAI_LITELLM_MASTER_KEY), then Check again.
LiteLLM does not answer at <url> (litellm_unreachable, 502)The LiteLLM proxy is down or the address is wrong.Check the proxy and its address, then Synchronize with LiteLLM.
"The LiteLLM admin API refused the request (status <status>)." (litellm_rejected, 502)LiteLLM rejected the team call.Check the master key and the LiteLLM logs, then synchronize again.
"An LLM project with this name already exists." (llm_project_exists, 409)Duplicate name in the tenant.Choose another Name.
"Tenant project not found." (tenant_project_not_found, 404)The linked Tenant project does not exist.Pick a project of the list, or leave the field empty.
"Invalid quota: <reason>." (invalid_quota, 422)A quota value is out of bounds.Fix the value named in the message.
"This project still has active keys: revoke them first." (llm_project_has_keys, 409)Keys not revoked on the project.Revoke them in LiteLLM keys.
"This action needs a tenant: sign in on a tenant or send X-Tenant." (tenant_required)No tenant context.Sign in on a tenant.
Error panel forbidden (403)Role other than tenant_admin or platform_admin.Ask a tenant administrator.