# AI backends

Roscoe needs an AI backend to execute `ai_agent`, `ai_judge`, and `consensus` nodes outside of MCP mode. There are two: the official Anthropic SDK (fast, in-process, requires an API key) and the Claude Code CLI (subprocess, no API key, requires `claude` on `PATH`). Roscoe auto-detects which to use at process start and caches the choice for the lifetime of the process.

Reach for this page when you're deciding how to authenticate, debugging an "AI node failed" error, or planning a CI run that doesn't have interactive `claude login` available.

## The two backends

### `SdkBackend`

Selected when `ANTHROPIC_API_KEY` is set in the environment. Calls the Anthropic API directly via the official SDK: no extra processes, no shell out. Fastest path; recommended for CI and production. Errors out if a response hits the `max_tokens` limit, which surfaces truncation rather than silently chopping output.

### `CliBackend`

Selected when `ANTHROPIC_API_KEY` is **not** set. Spawns:

```bash
claude -p --model <model> --output-format stream-json --verbose
```

with the prompt on stdin, then parses the newline-delimited stream for the final response text plus cost and usage from the terminal `result` event (and any `thinking` blocks the model emitted). The child is killed on `SIGTERM` at the node's timeout. Requires the [Claude Code CLI](https://claude.ai/code) to be installed and authenticated. A tool-enabled [`ai_agent`](/docs/nodes/ai-agent#tool-enabled-agents) node adds `--allowedTools <names>` and runs the child in the run's working directory.

## Auto-detection

```mermaid
flowchart TD
  A[getLlmBackend called] --> B{ANTHROPIC_API_KEY set?}
  B -- yes --> C[SdkBackend]
  B -- no --> D{claude --version OK?}
  D -- no --> E[Error: install Claude Code or set ANTHROPIC_API_KEY]
  D -- yes --> F[CliBackend]
```

The check runs once per process. The first AI node pays the validation cost; subsequent nodes reuse the cached backend.

### Tool-enabled agents always use the CLI

An `ai_agent` node with [`allowedTools`](/docs/nodes/ai-agent#tool-enabled-agents) set is the exception to the table above: it always runs on `CliBackend`, even when `ANTHROPIC_API_KEY` is set. The SDK has no built-in file/bash tools, so tool grants only work through the CLI. If the CLI isn't installed, such a node fails with a tool-specific message (setting an API key would not help, so it isn't suggested):

> This node has tools enabled, which requires the claude CLI, but it was not found. Install Claude Code (https://claude.ai/code) to run tool-enabled agents.

A node with _no_ `allowedTools` can also land on the CLI: when a single interpolated value exceeds 16 KiB, Roscoe spills it to a queryable file and auto-grants read-only `Read`/`Grep` so the model can query it, which routes the node through `CliBackend` (see [Oversized inputs](/docs/nodes/ai-agent#oversized-inputs)). If the CLI isn't available in that case there's no hard failure, but the value is head/tail truncated on the default backend instead.

## Errors you might see

### `claude CLI not found. Install Claude Code (https://claude.ai/code) or set ANTHROPIC_API_KEY.`

`ANTHROPIC_API_KEY` is unset and `claude --version` either failed or `claude` isn't on `PATH`. Fix it by installing Claude Code or by exporting an API key:

```bash
export ANTHROPIC_API_KEY=sk-ant-...
```

### `claude CLI is not authenticated. Run \`claude login\` and try again.`

`claude` is on `PATH` but the local install isn't logged in. Run:

```bash
claude login
```

This is the most common CI failure mode: CI containers rarely have an interactive login. Use `ANTHROPIC_API_KEY` in CI instead.

### `claude CLI timed out after <n>s for model "<model>"`

A single AI call ran past its configured timeout. A plain `ai_agent` or `ai_judge` node defaults to 300s (5 minutes); a `consensus`, `map`, or `round_robin` sub-call is capped at 120s by default unless the node sets its own `timeoutSeconds`. Either the model is overloaded, the prompt is huge, or the network is degraded. Re-run; if it persists, raise `timeoutSeconds` on the node, switch to a faster model in `roscoe.yaml`, or switch to `SdkBackend` by setting `ANTHROPIC_API_KEY`.

### `Model "<model>" hit the token limit — response was truncated`

`SdkBackend` aborted because the model returned `stop_reason: max_tokens`. Tighten the prompt, use a model with a larger output budget, or split the work across nodes.

## MCP mode caveat

When a workflow runs via the MCP server (started by Claude Code), `ai_agent` and `ai_judge` nodes do **not** use either backend: the runner pauses, returns a `pendingStep`, and Claude Code itself supplies the response via `advance_run`. See [/docs/running/mcp](/docs/running/mcp).

`consensus` is the exception. Even in MCP mode it calls the backend directly (N parallel SDK / CLI calls), so the quorum tally stays a single atomic gate rather than N pause-resume cycles. This means a workflow with a consensus node still needs one of the two backends available even when driven from Claude Code. `test_workflow` (the MCP verification tool) explicitly refuses to run a workflow with a consensus node when no backend is configured.

## See also

- [/docs/configuration/environment](/docs/configuration/environment) — `ANTHROPIC_API_KEY` and other process-start vars.
- [/docs/configuration/roscoe-yaml](/docs/configuration/roscoe-yaml) — the model allowlist a backend dispatches against.
- [/docs/running/mcp](/docs/running/mcp) — how MCP mode bypasses the backend for most AI nodes.