Skip to main content
Talk mode covers five runtime shapes:
  • Native macOS/iOS/Android Talk: local speech recognition, Gateway chat, and talk.speak TTS. Nodes advertise the talk capability and declare which talk.* commands they support.
  • iOS Talk (realtime): client-owned WebRTC for OpenAI realtime configs that select webrtc transport or omit transport. Explicit gateway-relay, provider-websocket, and non-OpenAI realtime configs stay on the Gateway-owned relay; non-realtime configs use the native speech loop.
  • Browser Talk: talk.client.create for client-owned webrtc/provider-websocket sessions, or talk.session.create for Gateway-owned gateway-relay sessions. managed-room is reserved for Gateway handoff and walkie-talkie rooms.
  • Android Talk (realtime): opt in with talk.realtime.mode: "realtime" and talk.realtime.transport: "gateway-relay". Otherwise Android stays on native speech recognition, Gateway chat, and talk.speak.
  • Transcription-only clients: talk.session.create({ mode: "transcription", transport: "gateway-relay", brain: "none" }), then talk.session.appendAudio, talk.session.cancelTurn, and talk.session.close for captions/dictation without an assistant voice response. One-shot uploaded voice notes still use the media understanding audio path.
Native Talk is a continuous loop: listen for speech, send the transcript to the model through the active session, wait for the response, then speak it via the configured Talk provider (talk.speak). Client-owned realtime Talk forwards provider tool calls through talk.client.toolCall instead of calling chat.send directly. While a realtime consult is active, clients can call talk.client.steer or talk.session.steer to classify spoken input as status, steer, cancel, or followup. Accepted steering queues into the active embedded run; rejected steering returns a reason such as no_active_run, not_streaming, or compacting. Transcription-only Talk emits the same Talk event envelope as realtime and STT/TTS sessions, but uses mode: "transcription" and brain: "none". All Talk sessions broadcast events on the talk.event channel; clients subscribe to it for partial/final transcript updates (transcript.delta/transcript.done) and other session telemetry.

​
Behavior (macOS)

  • Always-on overlay while Talk mode is enabled.
  • Listening → Thinking → Speaking phase transitions.
  • On a short pause (silence window), the current transcript is sent.
  • Replies are written to WebChat (same as typing).
  • Interrupt on speech (default on): if the user talks while the assistant is speaking, playback stops and the interruption timestamp is noted for the next prompt.

​
Voice directives in replies

The assistant can prefix a reply with a single JSON line to control voice:
Rules:
  • First non-empty line only; the JSON line is stripped before TTS playback.
  • Unknown keys are ignored.
  • once: true applies to the current reply only; without it, the voice becomes the new Talk mode default.
Supported keys: voice / voice_id / voiceId, model / model_id / modelId, speed, rate (WPM), stability, similarity, style, speakerBoost, seed, normalize, lang, output_format, latency_tier, once.

​
Config (~/.openclaw/openclaw.json)

talk.catalog exposes canonical provider ids and registry aliases, each provider’s valid modes/transports/brain strategies/realtime audio formats/capability flags, and the runtime-selected readiness result. First-party Talk clients should read that catalog instead of maintaining provider aliases locally; treat an older Gateway that omits group readiness as unverified rather than definitively unconfigured. Streaming transcription providers are discovered through talk.catalog.transcription; the current Gateway relay uses the Voice Call streaming provider config until a dedicated Talk transcription config surface ships.

​
macOS UI

  • Menu bar toggle: Talk
  • Config tab: Talk Mode group (voice id + interrupt toggle)
  • Overlay: the orb renders the universal talk waveform (shared with iOS, watchOS, and Android). Listening follows the live mic level, Speaking follows the actual TTS playback envelope, Thinking breathes softly. Click the orb to pause/resume, double-click to stop speaking, click X to exit Talk mode.

​
Android UI

  • Voice tab toggle: Talk
  • Manual Mic and Talk are mutually exclusive capture modes.
  • Manual Mic and realtime Talk prefer a connected Bluetooth Classic or BLE headset microphone; if it disconnects, the app requests another headset input or falls back to the default microphone, restoring the default preference once capture stops.
  • Manual Mic stops when the app leaves the foreground or the user leaves the Voice tab.
  • Talk Mode keeps running until toggled off or the node disconnects, using Android’s microphone foreground-service type while active.
  • Android supports pcm_16000, pcm_22050, pcm_24000, and pcm_44100 output formats for low-latency AudioTrack streaming.

​
Notes

  • Requires Speech + Microphone permissions.
  • Native Talk uses the active Gateway session and only falls back to history polling when response events are unavailable.
  • The gateway resolves Talk playback through talk.speak using the active Talk provider. Android falls back to local system TTS only when that RPC is unavailable.
  • macOS local MLX playback uses the bundled openclaw-mlx-tts helper when present, or an executable on PATH. Set OPENCLAW_MLX_TTS_BIN to point at a custom helper binary during development.
  • Voice directive value ranges (ElevenLabs): stability, similarity, and style accept 0..1; speed accepts 0.5..2; latency_tier accepts 0..4.