enterprise_ai.multimodal.realtime

Source: sdks/python/src/enterprise_ai/multimodal/realtime/__init__.py. Back to the Python SDK reference index.

note

This page is generated by tools/docs/gen_sdk_reference.py from the SDK sources and the OpenAPI text folders. Do not edit it by hand: change the source and regenerate.

Class RealtimeError

An error event received on the realtime socket.

EaiError

Model RealtimeEvent

One decoded server event.

Fields:

Name	Type	Required	Description
type	str	yes	Type of the event, for example transcript.delta or response.done (unknown when absent).
data	dict[str, Any]	no	Decoded fields of the server event, besides its type.

Attributes:

Name	Type	Required	Description
audio	bytes	yes	Decoded audio of a response.audio.delta event.
text	str	yes	Text of a transcript.delta event.

Class RealtimeConnection

An open realtime WebSocket (async context manager).

Attributes:

Name	Type	Required	Description
headers		yes	HTTP headers sent when the socket opens.
raise_on_error		yes	Whether a server error event raises RealtimeError.
url		yes	WebSocket URL, with ?token= once a token was obtained.
usage	dict[str, Any]	yes	Last usage reported by a usage.update event, {} before.
ws	Any	yes	The open socket, None before opening and after close.

Constructor RealtimeConnection

RealtimeConnection(url: str, headers: dict[str, str], raise_on_error: bool = True, connector: Any = None, token_provider: Callable[[], Any] | None = None)

Prepare the connection; the socket opens on async with.

Name	Type	Default	Required	Description
url	str		yes	WebSocket URL of the session.
headers	dict[str, str]		yes	HTTP headers sent when the socket opens (authentication, tenant).
raise_on_error	bool	True	no	Raise RealtimeError on a server error event (default True).
connector	Any	None	no	Coroutine function opening the socket, default websockets.asyncio.client.connect.
token_provider	Callable[[], Any] | None	None	no	Called when the connection opens to get a new ws_token, added to url as ?token=.

RealtimeConnection.append_audio

async append_audio(pcm: bytes) -> None

input_audio.append with base64 encoded audio.

Name	Type	Default	Required	Description
pcm	bytes		yes	Raw audio bytes to send.

Returns: None.

RealtimeConnection.cancel_response

async cancel_response() -> None

response.cancel: stops the response in progress (response.cancelled).

This element takes no parameter.

Returns: None.

RealtimeConnection.close

async close() -> None

Close the socket; nothing happens when it is not open.

This element takes no parameter.

Returns: None.

RealtimeConnection.commit_audio

async commit_audio() -> None

input_audio.commit: ends the utterance (saves the recording when config.record is set).

This element takes no parameter.

Returns: None.

RealtimeConnection.create_response

async create_response(text: str, voice: str | None = None) -> None

response.create {text, voice?}: the server speaks text (response.created then audio).

Name	Type	Default	Required	Description
text	str		yes	Text the server speaks.
voice	str | None	None	no	Voice to use, optional; left out of the event when not given.

Returns: None.

RealtimeConnection.events

async events() -> AsyncIterator[RealtimeEvent]

Yield server events until the socket closes; error raises when raise_on_error.

This element takes no parameter.

Returns: AsyncIterator[RealtimeEvent]. A RealtimeEvent: One decoded server event.

Raises: RealtimeError.

RealtimeConnection.send

async send(event: dict[str, Any]) -> None

Send a client event.

Name	Type	Default	Required	Description
event	dict[str, Any]		yes	Client event, sent as JSON (see client_event).

Returns: None.

RealtimeConnection.speech_started

async speech_started() -> None

input_audio.speech_started: barge-in, the server cancels the response being spoken.

This element takes no parameter.

Returns: None.

RealtimeConnection.update_session

async update_session(**settings: Any) -> None

session.update (voice, language, turn detection, instructions...).

Name	Type	Default	Required	Description
**settings	Any		no	Session settings sent as session, for example voice and language.

Returns: None.

Class RealtimeClient

/v1/realtime.

Resource

RealtimeClient.connect

connect(session_id: str | dict[str, Any], ws_token: str | None = None, raise_on_error: bool = True, connector: Any = None) -> RealtimeConnection

Open the realtime socket (use as async with); the server requires a single use ws_token.

Name	Type	Default	Required	Description
session_id	str | dict[str, Any]		yes	Session id, or the create_session or create_token answer (its ws_token is then used).
ws_token	str | None	None	no	Single use token; without one, a new token is requested when the connection opens.
raise_on_error	bool	True	no	Raise RealtimeError on a server error event (default True).
connector	Any	None	no	Coroutine function opening the socket, default websockets.asyncio.client.connect; replace it in tests.

Returns: RealtimeConnection. A RealtimeConnection: An open realtime WebSocket (async context manager).

RealtimeClient.create_session

create_session(pipeline: str = 'echo', room: str | None = None, ttl_seconds: int | None = None, config: dict[str, Any] | None = None) -> Any

Open a realtime session. Opens a session (201) and returns it with session_id, livekit (url, token, room, configured) and the WebSocket access (ws_url, ws_token, ws_token_expires_at). Needs tenant_admin, developer or agent. Checks: pipeline is echo or stt (400 invalid_request); config.vad and config.dlp_mode are valid (422 parameter_invalid); an agent caller acts for on_behalf_of, a user may only open a session for itself (403 identity_mismatch); LiveKit credentials are configured; the policy realtime.session.create is evaluated. The room defaults to a name derived from the tenant and the session. Audited as realtime.session.create.

Route: /v1/realtime/sessions (POST).

Name	Type	Default	Required	Description
pipeline	str	'echo'	no	echo (default) or stt.
room	str | None	None	no	LiveKit room name (1 to 128 letters, digits, ., _, -). Default: derived from the tenant and the session.
ttl_seconds	int | None	None	no	Lifetime of the LiveKit token, in seconds (60 to 86400, default 3600).
config	dict[str, Any] | None	None	no	Session configuration: dlp_mode (mask default, tokenize or block), vad (enabled boolean, threshold in (0, 1]), record and pipeline options.

Returns: Any. Decoded JSON answer of the operation "Open a realtime session".

Raises: the typed errors of the transport (Errors).

RealtimeClient.create_token

create_token(session_id: str) -> Any

Get a new WebSocket token. New single use WebSocket token for an open session (reconnection), read under RLS: ws_url, ws_token, ws_token_expires_at. 409 session_ended; 404 not_found. Needs tenant_admin, developer or agent. Audited as realtime.session.token.

Route: /v1/realtime/sessions/{session_id}/token (POST).

Name	Type	Default	Required	Description
session_id	str		yes	Identifier (UUID) of the realtime session.

Returns: Any. Decoded JSON answer of the operation "Get a new WebSocket token".

Raises: the typed errors of the transport (Errors).

RealtimeClient.end_session

end_session(session_id: str) -> Any

End a realtime session. Ends the session and records its usage event (voice_live: audio seconds, input and output tokens, measured cost). Ending a session already ended returns it unchanged. Needs tenant_admin, developer or agent. Audited as realtime.session.end.

Route: /v1/realtime/sessions/{session_id}/end (POST).

Name	Type	Default	Required	Description
session_id	str		yes	Identifier (UUID) of the realtime session.

Returns: Any. Decoded JSON answer of the operation "End a realtime session".

Raises: the typed errors of the transport (Errors).

RealtimeClient.get_session

get_session(session_id: str) -> Any

Get a realtime session. Returns the session: room, identities, agent, pipeline, configuration, status, audio seconds, tokens and usage. 404 not_found.

Route: /v1/realtime/sessions/{session_id} (GET).

Name	Type	Default	Required	Description
session_id	str		yes	Identifier (UUID) of the realtime session.

Returns: Any. Decoded JSON answer of the operation "Get a realtime session".

Raises: the typed errors of the transport (Errors).

RealtimeClient.list_consents

list_consents(session_id: str) -> Any

List the consents of a session. Consents recorded for the session, in capture order, with their revocation when revoked. One page only.

Route: /v1/realtime/sessions/{session_id}/consents (GET).

Name	Type	Default	Required	Description
session_id	str		yes	Identifier (UUID) of the realtime session.

Returns: Any. Decoded JSON answer of the operation "List the consents of a session".

Raises: the typed errors of the transport (Errors).

RealtimeClient.record_consent

record_consent(session_id: str, purpose: str, retention_days: int, recording_allowed: bool | None = None, transcription_allowed: bool | None = None, method: str | None = None, participant_identity: str | None = None) -> Any

Record a participant consent. Consent of one participant (purpose, recording, transcription, retention, method), captured by the caller (201). The participant defaults to the participant of the session. The method must be verbal, click, written or api (422 parameter_invalid); the session must not be ended (409 session_ended); the policy realtime.consent.record is evaluated. Needs tenant_admin, developer or agent. Audited as realtime.consent.record.

Route: /v1/realtime/sessions/{session_id}/consents (POST).

Name	Type	Default	Required	Description
session_id	str		yes	Identifier (UUID) of the realtime session.
purpose	str		yes	Purpose the participant agrees to (1 to 500 characters).
retention_days	int		yes	How long transcripts and recordings are kept, in days (1 to 3650).
recording_allowed	bool | None	None	no	Whether the audio may be recorded. Default false.
transcription_allowed	bool | None	None	no	Whether the speech may be transcribed and stored. Default false.
method	str | None	None	no	How the consent was captured: verbal, click, written or api (default).
participant_identity	str | None	None	no	Identity of the participant (1 to 128 letters, digits, ., _, :, @, -). Default: the participant of the session.

Returns: Any. Decoded JSON answer of the operation "Record a participant consent".

Raises: the typed errors of the transport (Errors).

RealtimeClient.revoke_consent

revoke_consent(session_id: str, consent_id: str) -> Any

Revoke a consent. Revocation takes effect on the next audio event: transcription and recording are refused again. A consent already revoked or unknown answers 404 not_found. Needs tenant_admin, developer or agent. Audited as realtime.consent.revoke.

Route: /v1/realtime/sessions/{session_id}/consents/{consent_id}/revoke (POST).

Name	Type	Default	Required	Description
session_id	str		yes	Identifier (UUID) of the realtime session.
consent_id	str		yes	Identifier (UUID) of the consent.

Returns: Any. Decoded JSON answer of the operation "Revoke a consent".

Raises: the typed errors of the transport (Errors).

RealtimeClient.transcripts

transcripts(session_id: str) -> Any

List the transcripts of a session. Stored transcripts, already filtered by the DLP (the raw transcript is never stored), with turn, consent, participant, DLP findings and expiry. Needs tenant_admin, developer, auditor or security_officer.

Route: /v1/realtime/sessions/{session_id}/transcripts (GET).

Name	Type	Default	Required	Description
session_id	str		yes	Identifier (UUID) of the realtime session.

Returns: Any. Decoded JSON answer of the operation "List the transcripts of a session".

Raises: the typed errors of the transport (Errors).

RealtimeClient.turns

turns(session_id: str) -> Any

List the turns of a session. Turns with the participant, the user, the agent, the trace id, the status (a barge-in records cancelled) and the measured latency, in order. One page only.

Route: /v1/realtime/sessions/{session_id}/turns (GET).

Name	Type	Default	Required	Description
session_id	str		yes	Identifier (UUID) of the realtime session.

Returns: Any. Decoded JSON answer of the operation "List the turns of a session".

Raises: the typed errors of the transport (Errors).

RealtimeClient.ws_url

ws_url(session_id: str | dict[str, Any], ws_token: str | None = None) -> str

URL of WebSocket /v1/realtime/ws/{session_id}?token=<ws_token>.

Name	Type	Default	Required	Description
session_id	str | dict[str, Any]		yes	Session id, or the create_session answer (its session_id and ws_token are read).
ws_token	str | None	None	no	Single use token added as ?token=, optional.

Returns: str. wss:// URL (ws:// when the base URL is not HTTPS), with the token when known.

Functions

client_event

client_event(event_type: str, **payload: Any) -> dict[str, Any]

Build a client event with an event_id.

Name	Type	Default	Required	Description
event_type	str		yes	One of session.update, input_audio.append, input_audio.commit, input_audio.speech_started, response.create, response.cancel; any other raises ValueError.
**payload	Any		no	Fields of the event, merged next to type and event_id.

Returns: dict[str, Any]. Event dict with type, a new event_id (evt_ prefix) and the payload.

Raises: ValueError.

parse_event

parse_event(message: str | bytes) -> RealtimeEvent

Decode a server message.

Name	Type	Default	Required	Description
message	str | bytes		yes	JSON message received on the socket, text or bytes.

Returns: RealtimeEvent. A RealtimeEvent: One decoded server event.