Events you get

The session sends these as JSON on the nuance_live topic. Every event has a type and an event_id. Times (start_ms, end_ms, offset_ms, cut_ms) are milliseconds on the session's timeline, where 0 is when it started.

Ignore event types and fields you don't know: new ones are added.

The sessionCopy Anchor Link

session.started comes once the persona can hear the user. Its session describes what runs:

• session.id is live_ and the conversation's id.
• session.expires_at is when the session will end, in Unix seconds.
• session.delegation is the delegation as the session runs it.
• session.runtime_tools is the persona's own tools as the session runs them: always stay_quiet, and end_call and call_time when you named them.

A room never echoes your prompts or tool schemas: instructions is null, and each tool is named without its schema. Keep the definitions you minted.

WordsCopy Anchor Link

session.input_transcript.delta and session.output_transcript.delta carry delta, start_ms and end_ms. Deltas carry their own spaces, so join them as they come.

{ "type": "session.output_transcript.delta", "event_id": "evt_88", "delta": " It's light rain", "start_ms": 41230, "end_ms": 41890 }

• The user's words include words they cut off. A card number, a card's security code, a one-time code or a Social Security number the user reads aloud comes as [redacted].
• The persona's words come as it says them; its audio is on the room's audio track.
• session.input_transcript.done is the user's turn as the persona's reply to it understood it, once that reply opens: transcript is the turn's settled words, which may differ from the live ones (the deltas joined), with start_ms and end_ms. Use it as the turn's record. A turn can get a later one, when the persona's reply ended before it said a word and the next reply opens on the whole turn: the later one's replaces_event_id names the event_id of the one it replaces, so show it in that one's place. Typed words are in neither event.

The end of a replyCopy Anchor Link

session.output_audio.done says the audio of the persona's reply has ended, at the time its last audio plays on the track: wait for it before acting on what the persona is saying, such as hanging up after a goodbye.

Treat a reason you don't know as an end too. It also carries start_ms and end_ms, and delegation_id and response_id when the reply was opened to say a backend's answer. A reply that said nothing gets none. The persona's replies come without response.created or response.done events: wait for session.output_audio.done, never for the persona's next words, since it may stay quiet until the user speaks.

Hand-offsCopy Anchor Link

session.delegation.created says the persona handed work off: delegation.id (del_1, …), delegation.target (client or responses) and offset_ms, where on the timeline it did. Under a responses delegation, response.event then carries the backend's work, with the task's delegation_id and the backend's own event nested as event. See Delegation and tools.

Quiet holdsCopy Anchor Link

session.quiet.started and session.quiet.ended report a quiet hold: the persona's (by: "agent", it chose to stay quiet) or yours (by: "client", session.floor.hold).

• started carries seconds until it runs out (null: until it's released or the user speaks), ends_on_speech, your reason and offset_ms.
• ended carries cause: expired, speech (the user's words), released, replaced (by a newer hold) or session_ended.
• offset_ms is null when the time isn't known.

Errors and noticesCopy Anchor Link

{ "type": "error", "event_id": "evt_90", "error": { "type": "invalid_request_error", "code": "command_refused", "message": "the agent is not talking", "client_event_id": "stop_1" } }

error.type is invalid_request_error, or server_error for a failure of ours. error.code says what happened, error.param names the field when one is at fault, and error.client_event_id names the event of yours it answers. Most errors aren't fatal: the session goes on. An error instead of session.started means the session was refused at the start; error.message names the setting. Limits and errors lists the codes you'll meet.

info is a notice, with code and message, such as delegation_cancelled for a task the session withdrew.

The session's endCopy Anchor Link

session.closed is the last event, with reason, and, for a policy or age-check end, message: the words to show your user, the same for every end of its kind.

The events route's ended comes after it. If the room's connection is stuck, only ended comes. Either way, leave the room.