Local model services

models.providers.<id>.localService starts a provider-owned local model server on demand. When a model or embedding request selects that provider, OpenClaw probes the health endpoint, starts the process if it is down, waits for readiness, then sends the request. Use it to avoid keeping expensive local servers running all day.

How it works

1. A model or embedding request resolves to a configured provider.
2. If that provider has localService, OpenClaw probes healthUrl.
3. On a successful probe, OpenClaw uses the already-running server.
4. On a failed probe, OpenClaw spawns command with args.
5. OpenClaw polls the health endpoint until readyTimeoutMs expires.
6. The request goes through the normal model or embedding transport.
7. If OpenClaw started the process and idleStopMs is set, it stops the process after the last in-flight request has been idle that long.

OpenClaw does not install launchd, systemd, Docker, or any daemon for this. The server is a plain child process of whichever OpenClaw process first needed it.

Startup is serialized per configured provider and command/argument/env set, so concurrent chat and embedding requests for the same service do not spawn duplicate servers. Each request holds its own lease until response handling completes, so idle shutdown waits for every in-flight model and embedding request. Configured provider aliases remain distinct: two aliases can point at different GPU hosts without collapsing onto the same Ollama, LM Studio, or OpenAI-compatible adapter id.

If another OpenClaw process already has a healthy server at the same healthUrl, this process reuses it without adopting it (each process only manages the child it personally started). Startup and exit logs include bounded, redacted child-output tails plus timing and exit details; configured environment values are never emitted.

Managed llama.cpp

The official llama.cpp provider generates this shape automatically. Its guided setup installs a pinned, verified llama-server, writes an absolute command and router preset, selects a free loopback port, and stores the resulting baseUrl and localService config. Chat and local embeddings lease the same managed router through the normal OpenAI-compatible transports.

Do not copy a generated command path between machines. Run llama.cpp setup on each Gateway host so OpenClaw selects and verifies the matching platform build. See llama.cpp Provider.

Config shape

{
  models: {
    providers: {
      local: {
        baseUrl: "http://127.0.0.1:8000/v1",
        apiKey: "local-model",
        api: "openai-completions",
        timeoutSeconds: 300,
        localService: {
          command: "/absolute/path/to/server",
          args: ["--host", "127.0.0.1", "--port", "8000"],
          cwd: "/absolute/path/to/working-dir",
          env: { LOCAL_MODEL_CACHE: "/absolute/path/to/cache" },
          healthUrl: "http://127.0.0.1:8000/v1/models",
          readyTimeoutMs: 180000,
          idleStopMs: 0,
        },
        models: [
          {
            id: "my-local-model",
            name: "My Local Model",
            reasoning: false,
            input: ["text"],
            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
            contextWindow: 131072,
            maxTokens: 8192,
          },
        ],
      },
    },
  },
}

Set timeoutSeconds on the provider entry (not localService) so slow cold starts and long generations do not hit the default model request timeout. Set an explicit healthUrl whenever your server exposes readiness somewhere other than /models on the base URL.

Fields

Field	Required	Description
command	yes	Absolute executable path. No shell PATH lookup.
args	no	Process arguments. No shell expansion, pipes, globbing, or quoting.
cwd	no	Working directory for the process.
env	no	Environment variables merged over the OpenClaw process environment.
healthUrl	no	Readiness URL. Defaults to baseUrl with /models appended (http://127.0.0.1:8000/v1 becomes http://127.0.0.1:8000/v1/models).
readyTimeoutMs	no	Startup readiness deadline. Default: 120000.
idleStopMs	no	Idle shutdown delay for an OpenClaw-started process. 0 or omitted keeps it alive until OpenClaw exits.

llmman example

llmman is a custom OpenAI-compatible /v1 backend, so the same localService API works with an llmman provider entry. It listens on 127.0.0.1:17434 by default; LLMMAN_HOST overrides the bind address, while LLMMAN_LLM_LIBRARY overrides GPU auto-detection. Its API has no authentication, so keep the default loopback bind unless a trusted network boundary restricts access.

{
  agents: {
    defaults: {
      model: { primary: "llmman/gemma4" },
    },
  },
  models: {
    mode: "merge",
    providers: {
      llmman: {
        baseUrl: "http://127.0.0.1:17434/v1",
        apiKey: "llmman-local",
        api: "openai-completions",
        timeoutSeconds: 300,
        localService: {
          command: "/opt/homebrew/bin/llmman",
          args: ["serve", "gemma4"],
          env: { LLMMAN_CONTEXT_LENGTH: "65536" },
          healthUrl: "http://127.0.0.1:17434/v1/models",
          readyTimeoutMs: 180000,
          idleStopMs: 0,
        },
        models: [
          {
            id: "gemma4",
            name: "Gemma 4 (llmman)",
            reasoning: false,
            input: ["text"],
            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
            contextWindow: 65536,
            maxTokens: 4096,
          },
        ],
      },
    },
  },
}

Replace command with the result of which llmman on the machine running OpenClaw. Full llmman setup: llmman.

ds4 example

{
  models: {
    providers: {
      ds4: {
        baseUrl: "http://127.0.0.1:18000/v1",
        apiKey: "ds4-local",
        api: "openai-completions",
        timeoutSeconds: 300,
        localService: {
          command: "<DS4_DIR>/ds4-server",
          args: [
            "--model",
            "<DS4_DIR>/ds4flash.gguf",
            "--host",
            "127.0.0.1",
            "--port",
            "18000",
            "--ctx",
            "32768",
            "--tokens",
            "128",
          ],
          cwd: "<DS4_DIR>",
          healthUrl: "http://127.0.0.1:18000/v1/models",
          readyTimeoutMs: 300000,
          idleStopMs: 0,
        },
        models: [],
      },
    },
  },
}

Full setup, context sizing, and verification commands: ds4.

Related

Local models

Local model setup, provider choices, and safety guidance.

llmman

Run OpenClaw through the llmman OpenAI-compatible local server.