enterprise_ai.llm.gateway
Source: sdks/python/src/enterprise_ai/llm/gateway/__init__.py. Back to the Python SDK reference index.
This page is generated by tools/docs/gen_sdk_reference.py from the SDK sources and the OpenAPI text folders. Do not edit it by hand: change the source and regenerate.
Class EffortAdjustedWarning
Emitted when the requested effort is mapped to another level by the model policy.
UserWarning
Class ChatStream
Iterator of chat.completion.chunk dicts; :attr:accumulator holds the merged message.
Attributes:
| Name | Type | Required | Description |
|---|---|---|---|
accumulator | ChatAccumulator | yes | Rebuild the assistant message (content, reasoning, tool calls, usage) from chunks. |
Constructor ChatStream
ChatStream(events: Iterator[Any]) -> None
Wrap an iterator of Server-Sent Events into a stream of chat chunks.
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
events | Iterator[Any] | yes | Iterator of the Server-Sent Events of /v1/gateway/chat/completions. |
ChatStream.final
final() -> dict[str, Any]
Consume the rest of the stream and return the completion.
This element takes no parameter.
Returns: dict[str, Any]. Completion rebuilt by the accumulator, in the chat.completion format.
ChatStream.text
text() -> Iterator[str]
Yield only the text deltas.
This element takes no parameter.
Returns: Iterator[str]. Iterator of the non empty text deltas; the chunks are still merged into accumulator.
Class AsyncChatStream
Async counterpart of :class:ChatStream.
Attributes:
| Name | Type | Required | Description |
|---|---|---|---|
accumulator | ChatAccumulator | yes | Rebuild the assistant message (content, reasoning, tool calls, usage) from chunks. |
Constructor AsyncChatStream
AsyncChatStream(events: AsyncIterator[Any]) -> None
Wrap an async iterator of Server-Sent Events into a stream of chat chunks.
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
events | AsyncIterator[Any] | yes | Async iterator of the Server-Sent Events of /v1/gateway/chat/completions. |
AsyncChatStream.final
async final() -> dict[str, Any]
Consume the rest of the stream and return the completion.
This element takes no parameter.
Returns: dict[str, Any]. Completion rebuilt by the accumulator, in the chat.completion format.
Class GatewayClient
/v1/gateway/*.
Resource
Attributes:
| Name | Type | Required | Description |
|---|---|---|---|
registry | yes | Model registry cache used by precheck, or None. |
Constructor GatewayClient
GatewayClient(transport: Any, client: Any = None, registry: ModelRegistryCache | None = None) -> None
Create the gateway sub-client; with a registry, chat checks the effort before sending.
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
transport | Any | yes | Transport shared by the sub-clients of one client. | |
client | Any | None | no | Parent client. |
registry | ModelRegistryCache | None | None | no | Model registry cache used for the client side effort check, or None to skip it. |
GatewayClient.chat
chat(model: str, messages: list[dict[str, Any]], effort: str | Effort | None = None, tools: list[dict[str, Any]] | None = None, tool_choice: Any = None, response_format: Any = None, stream: bool = False, eai: dict[str, Any] | None = None, precheck: bool = True, raw: bool = False, **params: Any) -> Any
Call a chat model. OpenAI compatible chat completion through every control of the gateway; with stream: true the answer is Server-Sent Events, with the cost guardrails checked during the stream. Requires a tenant principal (403 platform_only) whose credential carries the scope models:invoke.
Route: /v1/gateway/chat/completions (POST).
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
model | str | yes | Only cards whose model pattern matches this model id (case insensitive glob). | |
messages | list[dict[str, Any]] | yes | Chat messages {role, content} sent to the model. | |
effort | str | Effort | None | None | no | Reasoning effort (none, low, medium, high, max), sent in eai.effort. |
tools | list[dict[str, Any]] | None | None | no | Tools its external agents may invoke (names or patterns such as crm.*). |
tool_choice | Any | None | no | OpenAI tool_choice: auto, none, required or a named tool. |
response_format | Any | None | no | OpenAI response_format of the answer, for example a JSON schema. |
stream | bool | False | no | Stream the answer as Server-Sent Events and return a ChatStream (default False). |
eai | dict[str, Any] | None | None | no | eai extension fields (agent_id, session_id, run_id, task_id, data_classification, purpose, region_allow, fallback_policy); an unknown field raises ValueError. |
precheck | bool | True | no | Check the effort against the model registry before sending (default True; skipped on the async transport). |
raw | bool | False | no | Return an ApiResponse (data, status, headers) instead of the decoded body. |
**params | Any | no | Extra fields merged into the request. |
Returns: Any. Decoded JSON answer of the operation "Call a chat model".
Raises: the typed errors of the transport (Errors).
GatewayClient.embeddings
embeddings(model: str, input: str | list[str], **params: Any) -> Any
Compute embeddings. OpenAI compatible embeddings through the same controls as chat (kill switches, allow lists, classification, DLP on each input, cost guardrails, policy decision, audit). The model must declare the embeddings capability and, when dimensions is given, support it (422 capability_not_supported, 422 embedding_dimensions_invalid). Requires a tenant principal with the scope models:invoke.
Route: /v1/gateway/embeddings (POST).
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
model | str | yes | Only cards whose model pattern matches this model id (case insensitive glob). | |
input | str | list[str] | yes | Text or list of texts to embed. | |
**params | Any | no | Extra fields merged into the request. |
Returns: Any. Decoded JSON answer of the operation "Compute embeddings".
Raises: the typed errors of the transport (Errors).
GatewayClient.estimate
estimate(model: str, messages: list[dict[str, Any]] | None = None, effort: str | None = None, max_output_tokens: int | None = None, **params: Any) -> Any
Estimate a gateway call. Runs every preflight control of the gateway without calling the model and returns the resolved model, decision id, input tokens, maximum output tokens, estimator, estimated cost, cost limit, fallbacks, warnings and context plan. A request refused by a control answers with the same error as the real call. Requires a tenant principal with the scope models:invoke.
Route: /v1/gateway/estimate (POST).
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
model | str | yes | Only cards whose model pattern matches this model id (case insensitive glob). | |
messages | list[dict[str, Any]] | None | None | no | Chat messages of the planned call. |
effort | str | None | None | no | Reasoning effort to price, sent in eai.effort. |
max_output_tokens | int | None | None | no | Maximum output tokens assumed by the estimate. |
**params | Any | no | Extra fields merged into the request. |
Returns: Any. Decoded JSON answer of the operation "Estimate a gateway call".
Raises: the typed errors of the transport (Errors).
GatewayClient.images
images(model: str, prompt: str, size: str | None = None, n: int = 1, **params: Any) -> Any
Generate images. Image generation through the image adapter in the OpenAI format. The prompt goes through the DLP of the tenant (422 dlp_blocked), the kill switch, classification, secret isolation and audit apply, and the cost is reserved by number of images; text fields of the answer (revised_prompt) are scanned too. Requires a tenant principal with the scope models:invoke. Without a configured adapter: 501 adapter_not_available.
Route: /v1/gateway/images/generations (POST).
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
model | str | yes | Only cards whose model pattern matches this model id (case insensitive glob). | |
prompt | str | yes | Description of the image to generate. | |
size | str | None | None | no | Size of the image, such as 1024x1024; the model default when omitted. |
n | int | 1 | no | Number of images (1 to 8, default 1). |
**params | Any | no | Extra fields merged into the request. |
Returns: Any. Decoded JSON answer of the operation "Generate images".
Raises: the typed errors of the transport (Errors).
GatewayClient.precheck
precheck(model: str, effort: str | Effort | None) -> EffortResolution | None
Client side effort check from the registry (None when the model is not in the registry).
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
model | str | yes | Identifier of the model to look up in the registry. | |
effort | str | Effort | None | yes | Requested reasoning effort; None skips the check. |
Returns: EffortResolution \| None. A EffortResolution: Result of :func:resolve_effort.
GatewayClient.rerank
rerank(model: str, query: str, documents: list[str], top_n: int | None = None, **params: Any) -> Any
Rerank documents. Reranks documents against a query through the same controls as chat (DLP on the query, cost guardrails, policy decision, audit). The model must declare the rerank capability (422 capability_not_supported). Requires a tenant principal with the scope models:invoke.
Route: /v1/gateway/rerank (POST).
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
model | str | yes | Only cards whose model pattern matches this model id (case insensitive glob). | |
query | str | yes | Query string parameters of the route. | |
documents | list[str] | yes | Documents to sort by relevance to the query. | |
top_n | int | None | None | no | Number of best documents to return; all when omitted. |
**params | Any | no | Extra fields merged into the request. |
Returns: Any. Decoded JSON answer of the operation "Rerank documents".
Raises: the typed errors of the transport (Errors).
GatewayClient.speech
speech(model: str, input: str, voice: str | None = None, **params: Any) -> Any
Synthesize speech from text. Text to speech through the TTS adapter, in the OpenAI audio/speech format; the audio comes back as is (audio/*). The tenant kill switch, data classification and caller clearance, secret isolation and audit apply, and the cost is reserved from the quotas by number of characters. Requires a tenant principal with the scope models:invoke. Without a configured adapter: 501 adapter_not_available; adapter unreachable: 502 upstream_error.
Route: /v1/gateway/audio/speech (POST).
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
model | str | yes | Only cards whose model pattern matches this model id (case insensitive glob). | |
input | str | yes | Text to turn into speech. | |
voice | str | None | None | no | Voice name of the adapter. Default: the adapter default voice. |
**params | Any | no | Extra fields merged into the request. |
Returns: Any. Decoded JSON answer of the operation "Synthesize speech from text".
Raises: the typed errors of the transport (Errors).
GatewayClient.transcribe
transcribe(model: str, audio: bytes, filename: str = 'audio.wav', **params: Any) -> Any
Transcribe an audio file. Speech to text through the STT adapter, in the OpenAI audio/transcriptions format. Requires a tenant principal with the scope models:invoke.
Route: /v1/gateway/audio/transcriptions (POST).
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
model | str | yes | Only cards whose model pattern matches this model id (case insensitive glob). | |
audio | bytes | yes | Audio content to transcribe, as bytes sent in a multipart upload. | |
filename | str | 'audio.wav' | no | Name of the audio file sent with the multipart part (default audio.wav). |
**params | Any | no | Extra fields merged into the request. |
Returns: Any. Decoded JSON answer of the operation "Transcribe an audio file".
Raises: the typed errors of the transport (Errors).
GatewayClient.video
video(model: str, prompt: str, **params: Any) -> Any
Start a video generation. Starts a video generation job on the video adapter, with the same kill switch, classification, DLP of the prompt, cost reservation and audit as the other adapters; read the job with GET /v1/gateway/video/generations/{job_id}. Requires a tenant principal with the scope models:invoke. Without a configured adapter: 501 adapter_not_available.
Route: /v1/gateway/video/generations (POST).
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
model | str | yes | Only cards whose model pattern matches this model id (case insensitive glob). | |
prompt | str | yes | Description of the video to generate. | |
**params | Any | no | Extra fields merged into the request. |
Returns: Any. Decoded JSON answer of the operation "Start a video generation".
Raises: the typed errors of the transport (Errors).
GatewayClient.video_status
video_status(generation_id: str) -> Any
Read a video generation job. State and result of a video generation job, read from the video adapter with the same kill switch and audit as the other adapter calls. Requires a tenant principal (403 platform_only otherwise) whose credential carries the scope models:invoke. Without a configured video adapter: 501 adapter_not_available.
Route: /v1/gateway/video/generations/{job_id} (GET).
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
generation_id | str | yes | Identifier of the job returned by POST /v1/gateway/video/generations. |
Returns: Any. Decoded JSON answer of the operation "Read a video generation job".
Raises: the typed errors of the transport (Errors).
Class AsyncGatewayClient
Async gateway: the effort pre-check awaits the registry lookup.
GatewayClient
AsyncGatewayClient.chat
async chat(model: str, messages: list[dict[str, Any]], effort: str | Effort | None = None, tools: list[dict[str, Any]] | None = None, tool_choice: Any = None, response_format: Any = None, stream: bool = False, eai: dict[str, Any] | None = None, precheck: bool = True, raw: bool = False, **params: Any) -> Any
Call a chat model. OpenAI compatible chat completion through every control of the gateway; with stream: true the answer is Server-Sent Events, with the cost guardrails checked during the stream. Requires a tenant principal (403 platform_only) whose credential carries the scope models:invoke.
Route: /v1/gateway/chat/completions (POST).
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
model | str | yes | Only cards whose model pattern matches this model id (case insensitive glob). | |
messages | list[dict[str, Any]] | yes | Chat messages {role, content} sent to the model. | |
effort | str | Effort | None | None | no | Reasoning effort (none, low, medium, high, max), sent in eai.effort. |
tools | list[dict[str, Any]] | None | None | no | Tools its external agents may invoke (names or patterns such as crm.*). |
tool_choice | Any | None | no | OpenAI tool_choice: auto, none, required or a named tool. |
response_format | Any | None | no | OpenAI response_format of the answer, for example a JSON schema. |
stream | bool | False | no | Stream the answer as Server-Sent Events and return an AsyncChatStream (default False). |
eai | dict[str, Any] | None | None | no | eai extension fields (agent_id, session_id, run_id, task_id, data_classification, purpose, region_allow, fallback_policy); an unknown field raises ValueError. |
precheck | bool | True | no | Check the effort against the model registry before sending (default True). |
raw | bool | False | no | Return an ApiResponse (data, status, headers) instead of the decoded body. |
**params | Any | no | Extra fields merged into the request. |
Returns: Any. Decoded JSON answer of the operation "Call a chat model".
Raises: the typed errors of the transport (Errors).
AsyncGatewayClient.precheck_async
async precheck_async(model: str, effort: str | Effort | None) -> EffortResolution | None
Async effort pre-check using the models sub-client.
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
model | str | yes | Identifier of the model to look up in the registry. | |
effort | str | Effort | None | yes | Requested reasoning effort; None skips the check. |
Returns: EffortResolution \| None. A EffortResolution: Result of :func:resolve_effort.
Functions
build_eai
build_eai(effort: str | Effort | None, eai: dict[str, Any] | None) -> dict[str, Any] | None
Merge the eai extension, normalizing the effort.
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
effort | str | Effort | None | yes | Reasoning effort, normalized into eai.effort when given. | |
eai | dict[str, Any] | None | yes | eai fields given by the caller; only the known eai fields are accepted. |
Returns: dict[str, Any] \| None. Merged eai dict, or None when empty; raises ValueError on an unknown field.
Raises: ValueError.