> ## Documentation Index
> Fetch the complete documentation index at: https://docs2.openclaw.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Plugin runtime media helpers

Speech, media understanding, generation, web search, and the low-level media utilities. Part of the [Plugin runtime helpers](https://docs2.openclaw.ai/plugins/sdk-runtime.md) reference.

## FFmpeg command discovery

Official plugin sources use `resolveFfmpegBin` from
the private `openclaw/plugin-sdk/media-ffmpeg` runtime. It resolves the same
trusted system paths as the media runtime and throws an installation hint when
FFmpeg is unavailable. This narrow entry point keeps media generation and agent
runtimes out of source audio worker startup. It has a JavaScript-only host export;
its declarations are excluded from the package. Third-party plugins retain the
existing `resolveFfmpegBin` export from `openclaw/plugin-sdk/media-runtime`.

The official plugin publication builder emits `media-runtime` for this private
import, including worker entries, so published plugins keep working on supported
hosts that predate `media-ffmpeg`. Both paths use the host's FFmpeg resolver.

## Realtime voice playback

Official audio workers use `createRealtimeVoiceOutputActivityTracker` and
`isRealtimeVoiceAudioAudible` from the private
`openclaw/plugin-sdk/realtime-voice-playback` runtime to avoid loading voice
session and agent runtimes from source. The publication builder emits the
existing `openclaw/plugin-sdk/realtime-voice` host import for these helpers,
including worker entries, to preserve supported hosts. The source facade has a
JavaScript-only host export; its declarations are excluded from the package.

## Media and generation namespaces

> **api.runtime.tts**
>
> Text-to-speech synthesis.
>
> ```typescript theme={"theme":{"light":"min-light","dark":"min-dark"}}
> // Standard TTS
> const clip = await api.runtime.tts.textToSpeech({
>   text: "Hello from OpenClaw",
>   cfg: api.config,
> });
>
> // Telephony-optimized TTS
> const telephonyClip = await api.runtime.tts.textToSpeechTelephony({
>   text: "Hello from OpenClaw",
>   cfg: api.config,
> });
>
> // List available voices
> const voices = await api.runtime.tts.listVoices({
>   provider: "elevenlabs",
>   cfg: api.config,
> });
> ```
>
> Uses core `tts` configuration and provider selection. Returns PCM audio buffer + sample rate. `textToSpeechStream` is also available for streaming synthesis.

> **api.runtime.mediaUnderstanding**
>
> Image, audio, and video analysis.
>
> ```typescript theme={"theme":{"light":"min-light","dark":"min-dark"}}
> // Describe an image
> const image = await api.runtime.mediaUnderstanding.describeImageFile({
>   filePath: "/tmp/inbound-photo.jpg",
>   cfg: api.config,
>   agentDir: "/tmp/agent",
> });
>
> // Prepare a capture limit before installing audio receive listeners.
> const budget = await api.runtime.mediaUnderstanding.resolveAudioInputBudget({
>   cfg: api.config,
> });
> // budget.enabled is false when audio understanding is disabled. Otherwise,
> // budget.maxBytes includes the container header and covers the largest fallback.
>
> // Transcribe audio
> const { text } = await api.runtime.mediaUnderstanding.transcribeAudioFile({
>   filePath: "/tmp/inbound-audio.ogg",
>   cfg: api.config,
>   mime: "audio/ogg", // optional, for when MIME cannot be inferred
> });
>
> // Describe a video
> const video = await api.runtime.mediaUnderstanding.describeVideoFile({
>   filePath: "/tmp/inbound-video.mp4",
>   cfg: api.config,
> });
>
> // Generic file analysis
> const result = await api.runtime.mediaUnderstanding.runFile({
>   filePath: "/tmp/inbound-file.pdf",
>   cfg: api.config,
> });
>
> // Structured image extraction through a specific provider/model.
> // Include at least one image; text inputs are supplemental context.
> // receiptImageBuffer is your own image bytes, not an SDK-provided value.
> const evidence = await api.runtime.mediaUnderstanding.extractStructuredWithModel({
>   provider: "codex",
>   model: "gpt-6-astra",
>   input: [
>     {
>       type: "image",
>       buffer: receiptImageBuffer,
>       fileName: "receipt.png",
>       mime: "image/png",
>     },
>     { type: "text", text: "Prefer the printed total over handwritten notes." },
>   ],
>   instructions: "Extract vendor, total, and searchable tags.",
>   schemaName: "receipt.evidence",
>   jsonSchema: {
>     type: "object",
>     properties: {
>       vendor: { type: "string" },
>       total: { type: "number" },
>       tags: { type: "array", items: { type: "string" } },
>     },
>     required: ["vendor", "total"],
>   },
>   cfg: api.config,
> });
> ```
>
> Returns `{ text: undefined }` when no output is produced (e.g. skipped input).
>
> `describeImageFileWithModel(...)` describes an already-known image through a specific provider/model, bypassing the default active-model resolution that `describeImageFile(...)` uses.

> **api.runtime.imageGeneration**
>
> Image generation.
>
> ```typescript theme={"theme":{"light":"min-light","dark":"min-dark"}}
> const result = await api.runtime.imageGeneration.generate({
>   prompt: "A robot painting a sunset",
>   cfg: api.config,
> });
>
> const providers = api.runtime.imageGeneration.listProviders({ cfg: api.config });
> ```

> **api.runtime.videoGeneration**
>
> Video generation, mirroring the image generation shape.
>
> ```typescript theme={"theme":{"light":"min-light","dark":"min-dark"}}
> const result = await api.runtime.videoGeneration.generate({
>   prompt: "A drone shot flying over a coastline at sunrise",
>   cfg: api.config,
> });
>
> const providers = api.runtime.videoGeneration.listProviders({ cfg: api.config });
> ```

> **api.runtime.musicGeneration**
>
> Music generation, mirroring the image generation shape.
>
> ```typescript theme={"theme":{"light":"min-light","dark":"min-dark"}}
> const result = await api.runtime.musicGeneration.generate({
>   prompt: "An upbeat lo-fi track for a coding session",
>   cfg: api.config,
> });
>
> const providers = api.runtime.musicGeneration.listProviders({ cfg: api.config });
> ```

> **api.runtime.webSearch**
>
> Web search.
>
> ```typescript theme={"theme":{"light":"min-light","dark":"min-dark"}}
> const providers = api.runtime.webSearch.listProviders({ config: api.config });
>
> const result = await api.runtime.webSearch.search({
>   config: api.config,
>   args: { query: "OpenClaw plugin SDK", count: 5 },
> });
> ```
>
> Search callers may supply a synchronous `assertCurrent` callback with `signal`
> to retain their authority through provider preparation. Guarded HTTP requests
> check it after transport preparation and before each request or redirect.
> A registered search provider using another transport must call the execution
> context's `assertCurrent` before each external side effect, after awaited
> preparation. The callback belongs to the caller and expires when the search
> finishes; providers must not replace it or treat its absence as permission.

> **api.runtime.media**
>
> Low-level media utilities.
>
> ```typescript theme={"theme":{"light":"min-light","dark":"min-dark"}}
> const webMedia = await api.runtime.media.loadWebMedia(url);
> const mime = await api.runtime.media.detectMime({ buffer });
> const kind = api.runtime.media.mediaKindFromMime("image/jpeg"); // "image"
> const isVoice = api.runtime.media.isVoiceCompatibleAudio({ fileName: filePath });
> const metadata = await api.runtime.media.getImageMetadata(buffer);
> const resized = await api.runtime.media.resizeToJpeg({ buffer, maxSide: 800, quality: 85 });
> ```
>
> QR helpers are exported by `openclaw/plugin-sdk/media-runtime`:
>
> ```typescript theme={"theme":{"light":"min-light","dark":"min-dark"}}
> import { resolvePreferredOpenClawTmpDir } from "openclaw/plugin-sdk/temp-path";
>
> const qr = await import("openclaw/plugin-sdk/media-runtime");
> const terminalQr = await qr.renderQrTerminal("https://openclaw.ai");
> const pngQr = await qr.renderQrPngBase64("https://openclaw.ai", {
>   scale: 6, // 1-12
>   marginModules: 4, // 0-16
> });
> const pngQrDataUrl = await qr.renderQrPngDataUrl("https://openclaw.ai");
> const tmpRoot = resolvePreferredOpenClawTmpDir();
> const pngQrFile = await qr.writeQrPngTempFile("https://openclaw.ai", {
>   tmpRoot,
>   dirPrefix: "my-plugin-qr-",
>   fileName: "qr.png",
> });
> ```