> ## Documentation Index > Fetch the complete documentation index at: https://docs2.openclaw.ai/llms.txt > Use this file to discover all available pages before exploring further. # Extending the QA stack ## Repo-backed seeds Seed assets live in `qa/`: * `qa/scenarios/index.yaml` * `qa/scenarios//*.yaml` Identity-sensitive channel changes use the isolated `channel-participant-identity-inspection` QA Channel flow. It drives a real ephemeral Gateway and mock provider, then inspects admitted runs with the same `openclaw audit --run ... --explain` JSON and human surfaces operators use. The flow includes lifecycle-owned restart and a row-count check for rejected pre-run ingress. These are intentionally in git so the QA plan is visible to both humans and the agent. `qa-lab` stays a generic YAML scenario runner. Each scenario YAML file is the source of truth for one test run and should define: * top-level `title` * `scenario` metadata * optional category, capability, lane, and risk metadata in `scenario` * docs and code refs in `scenario` * optional plugin requirements in `scenario` * optional gateway config patch in `scenario` * executable top-level `flow` for flow scenarios, or `scenario.execution.kind` / `scenario.execution.path` for Vitest and Playwright scenarios The reusable runtime surface that backs `flow` stays generic and cross-cutting. For example, YAML scenarios can combine transport-side helpers with browser-side helpers that drive the embedded Control UI through the Gateway `browser.request` seam without adding a special-case runner. Scenario files should be grouped by product capability rather than source tree folder. Keep scenario IDs stable when files move; use `docsRefs` and `codeRefs` for implementation traceability. The baseline list should stay broad enough to cover: * DM and channel chat * thread behavior * message action lifecycle * cron callbacks * memory recall * model switching * subagent handoff * repo-reading and docs-reading * one small build task such as Lobster Invaders ## Provider mock lanes `qa suite` has two local provider mock lanes: * `mock-openai` is the scenario-aware OpenClaw mock. It remains the default deterministic mock lane for repo-backed QA and parity gates. * `aimock` starts an AIMock-backed provider server for experimental protocol, fixture, record/replay, and chaos coverage. It is additive and does not replace the `mock-openai` scenario dispatcher. The scenario-aware mock answers Activity recap requests separately from agent turns. Recaps quote conversation text as data, so they cannot trigger scenario tools or enter the scenario request evidence returned by `/debug/requests`. For an IPv6 loopback server, run `pnpm openclaw qa mock-openai --host ::1`. The printed URL includes brackets, such as `http://[::1]:`; use that URL when configuring a client. QA Lab also brackets IPv6 hosts in its listen and advertised URLs. Pass the bare address to `--host`. Provider-lane implementation lives under `extensions/qa-lab/src/providers/`. Each provider owns its defaults, local server startup, gateway model config, auth-profile staging needs, and live/mock capability flags. Shared suite and gateway code routes through the provider registry instead of branching on provider names. ## Transport adapters `qa-lab` owns a generic transport seam for YAML QA scenarios. `qa-channel` is the synthetic default. `crabline` starts separate local provider servers and runs OpenClaw's normal channel plugins against their provider-shaped REST and streaming boundaries; it does not use Crabline's fixture-level local mock providers. `live` is reserved for real provider credentials and external channels. At the architecture level, the split is: * `qa-lab` owns generic scenario execution, worker concurrency, artifact writing, and reporting. * The transport adapter owns gateway config, readiness, inbound and outbound observation, transport actions, and normalized transport state. * YAML scenario files under `qa/scenarios/` define the test run; `qa-lab` provides the reusable runtime surface that executes them. ### Adapter shutdown and failure hooks The optional QA runner hooks follow one host-owned teardown order: 1. `cleanup()` stops new fixture actions while retaining credential authority and ownership of pending writes. Observers needed for final receipts can remain active through Gateway shutdown. Bound transport requests themselves; a scenario deadline does not settle an already-dispatched request. 2. The host stops the Gateway and confirms process shutdown. 3. `captureBeforeGatewayCleanup()` snapshots final native receipts before temporary Gateway state is removed. A successful capture runs once per Gateway lifetime. A thrown error retains the runtime evidence and fails teardown; it does not prevent post-stop fixture cleanup. 4. `cleanupAfterGatewayStop()` settles pending writes and owned fixture cleanup, then releases the credential lease. The host withholds this hook if Gateway shutdown is unconfirmed. Report failures rather than claiming successful cleanup. Omitted hooks perform no adapter-specific work; normal host teardown still runs. Errors are accumulated across cleanup phases. Keep unknown remote write outcomes explicit instead of inferring ownership or replaying writes. An optional `whenUnhealthy` promise **resolves**, rather than rejects, with a terminal error. The host aborts active flow admission; the adapter still owns settlement during cleanup. Adapters that omit it retain their explicit health checks and scenario deadlines. ### Adding a channel Adding a channel to the YAML QA system requires the channel implementation plus a scenario pack that exercises the channel contract. For smoke CI coverage, add the matching Crabline local provider server and expose it through the `crabline` driver. Do not add a new top-level QA command root when the shared `qa-lab` host can own the flow. `qa-lab` owns the shared host mechanics: * the `openclaw qa` command root * suite startup and teardown * worker concurrency * artifact writing * report generation * scenario execution * compatibility aliases for older `qa-channel` scenarios Runner plugins own the transport contract: * how `openclaw qa ` is mounted beneath the shared `qa` root * how the gateway is configured for that transport * how readiness is checked * how inbound events are injected * how outbound messages are observed * how transcripts and normalized transport state are exposed * how transport-backed actions are executed * which trusted module preloads the Gateway runtime needs * how transport-specific reset or cleanup is handled The minimum adoption bar for a new channel: 1. Keep `qa-lab` as the owner of the shared `qa` root. 2. Implement the transport runner on the shared `qa-lab` host seam. 3. Keep transport-specific mechanics inside the runner plugin or channel harness. 4. Mount the runner as `openclaw qa ` instead of registering a competing root command. Runner plugins should declare `qaRunners` in `openclaw.plugin.json` and export a matching `qaRunnerCliRegistrations` array from a lightweight `qa-runner-api.ts` surface. Installed plugins using the shipped `runtime-api.ts` contract remain supported through 2026-10-01 while authors migrate. Keep runner execution behind lazy entrypoints. An optional `adapterFactory` exposes the transport to shared scenarios without changing the command's existing scenario catalog. Same-channel partitions are serial unless the factory declares that every instance owns isolated credentials or disposable servers, Gateway state, and artifact paths. Module-backed flow scenarios additionally require `adapterFactory.supportsModuleFlows: true`; those factories must return adapters that implement `prepareFlow`. 5. Author or adapt YAML scenarios under the themed `qa/scenarios/` directories. 6. Use the generic scenario helpers for new scenarios. 7. Keep existing compatibility aliases working unless the repo is doing an intentional migration. The decision rule is strict: * If behavior can be expressed once in `qa-lab`, put it in `qa-lab`. * If behavior depends on one channel transport, keep it in that runner plugin or plugin harness. * If a scenario needs a new capability that more than one channel can use, add a generic helper instead of a channel-specific branch in `suite.ts`. * If a behavior is only meaningful for one transport, keep the scenario transport-specific and make that explicit in the scenario contract. ### Scenario helper names Preferred generic helpers for new scenarios: * `waitForTransportReady` * `waitForChannelReady` * `injectInboundMessage` * `injectOutboundMessage` * `waitForOutboundMessage` * `waitForNoTransportOutbound` * `getTransportSnapshot` * `readTransportMessage` * `readTransportTranscript` * `formatTransportTranscript` * `resetTransport` Compatibility aliases remain available for existing scenarios - `waitForQaChannelReady`, `waitForNoOutbound`, `formatConversationTranscript`, and `resetBus` - but new scenario authoring should use the generic names. Use the canonical `waitForOutboundMessage` for outbound checks instead of adding transport- or channel-specific outbound wait aliases.