AI backends

Roscoe needs an AI backend to execute ai_agent, ai_judge, and consensus nodes outside of MCP mode. There are two: the official Anthropic SDK (fast, in-process, requires an API key) and the Claude Code CLI (subprocess, no API key, requires claude on PATH). Roscoe auto-detects which to use at process start and caches the choice for the lifetime of the process.

Reach for this page when you're deciding how to authenticate, debugging an "AI node failed" error, or planning a CI run that doesn't have interactive claude login available.

The two backends#

SdkBackend#

Selected when ANTHROPIC_API_KEY is set in the environment. Calls the Anthropic API directly via the official SDK: no extra processes, no shell out. Fastest path; recommended for CI and production. Errors out if a response hits the max_tokens limit, which surfaces truncation rather than silently chopping output.

CliBackend#

Selected when ANTHROPIC_API_KEY is not set. Spawns:

claude -p --model <model> --output-format stream-json --verbose

with the prompt on stdin, then parses the newline-delimited stream for the final response text plus cost and usage from the terminal result event (and any thinking blocks the model emitted). The child is killed on SIGTERM at the node's timeout. Requires the Claude Code CLI to be installed and authenticated. A tool-enabled ai_agent node adds --allowedTools <names> and runs the child in the run's working directory.

Auto-detection#

flowchart TD
  A[getLlmBackend called] --> B{ANTHROPIC_API_KEY set?}
  B -- yes --> C[SdkBackend]
  B -- no --> D{claude --version OK?}
  D -- no --> E[Error: install Claude Code or set ANTHROPIC_API_KEY]
  D -- yes --> F[CliBackend]

The check runs once per process. The first AI node pays the validation cost; subsequent nodes reuse the cached backend.

Tool-enabled agents always use the CLI#

An ai_agent node with allowedTools set is the exception to the table above: it always runs on CliBackend, even when ANTHROPIC_API_KEY is set. The SDK has no built-in file/bash tools, so tool grants only work through the CLI. If the CLI isn't installed, such a node fails with a tool-specific message (setting an API key would not help, so it isn't suggested):

This node has tools enabled, which requires the claude CLI, but it was not found. Install Claude Code (https://claude.ai/code) to run tool-enabled agents.

A node with no allowedTools can also land on the CLI: when a single interpolated value exceeds 16 KiB, Roscoe spills it to a queryable file and auto-grants read-only Read/Grep so the model can query it, which routes the node through CliBackend (see Oversized inputs). If the CLI isn't available in that case there's no hard failure, but the value is head/tail truncated on the default backend instead.

Errors you might see#

claude CLI not found. Install Claude Code (https://claude.ai/code) or set ANTHROPIC_API_KEY.#

ANTHROPIC_API_KEY is unset and claude --version either failed or claude isn't on PATH. Fix it by installing Claude Code or by exporting an API key:

export ANTHROPIC_API_KEY=sk-ant-...

claude CLI is not authenticated. Run \claude login` and try again.`#

claude is on PATH but the local install isn't logged in. Run:

claude login

This is the most common CI failure mode: CI containers rarely have an interactive login. Use ANTHROPIC_API_KEY in CI instead.

claude CLI timed out after <n>s for model "<model>"#

A single AI call ran past its configured timeout. A plain ai_agent or ai_judge node defaults to 300s (5 minutes); a consensus, map, or round_robin sub-call is capped at 120s by default unless the node sets its own timeoutSeconds. Either the model is overloaded, the prompt is huge, or the network is degraded. Re-run; if it persists, raise timeoutSeconds on the node, switch to a faster model in roscoe.yaml, or switch to SdkBackend by setting ANTHROPIC_API_KEY.

Model "<model>" hit the token limit — response was truncated#

SdkBackend aborted because the model returned stop_reason: max_tokens. Tighten the prompt, use a model with a larger output budget, or split the work across nodes.

MCP mode caveat#

When a workflow runs via the MCP server (started by Claude Code), ai_agent and ai_judge nodes do not use either backend: the runner pauses, returns a pendingStep, and Claude Code itself supplies the response via advance_run. See /docs/running/mcp.

consensus is the exception. Even in MCP mode it calls the backend directly (N parallel SDK / CLI calls), so the quorum tally stays a single atomic gate rather than N pause-resume cycles. This means a workflow with a consensus node still needs one of the two backends available even when driven from Claude Code. test_workflow (the MCP verification tool) explicitly refuses to run a workflow with a consensus node when no backend is configured.

See also#

• /docs/configuration/environment — ANTHROPIC_API_KEY and other process-start vars.
• /docs/configuration/roscoe-yaml — the model allowlist a backend dispatches against.
• /docs/running/mcp — how MCP mode bypasses the backend for most AI nodes.