Deferred init (warm-pool)

A ready-to-run example is available here.

In warm-pool deployments server pods are booted before a user is matched to one. The pod starts in a dormant state — stateless services (tool preload, VSCode, etc.) come up normally, but all /api/* routes return 503 until POST /api/init delivers the per-user runtime configuration (credentials, workspace paths, session keys).

This pattern reduces cold-start latency for users while keeping per-user data out of the image.

State machine

dormant ──(POST /api/init)──▶ initializing ──▶ ready
   ▲                               │
   └───────────(on error)──────────┘

State	/health, /ready	GET /api/init	POST /api/init	/api/*
dormant	200	200 state: dormant	200 → starts init	503
initializing	200	200 state: initializing	400 (already running)	503
ready	200	200 state: ready	400 (already done)	live

When deferred_init is false (the default), the /api/init endpoints return 404 and all /api/* routes are live immediately.

Enabling dormant mode

Set the FC_DEFERRED_INIT environment variable when starting the server:

FC_DEFERRED_INIT=true FC_SECRET_KEY=<bootstrap-secret> python -m faheemcode.agent_server

The FC_SECRET_KEY value is used to authenticate POST /api/init via the X-Init-API-Key request header. The orchestrator already holds this key for encryption purposes, so no additional secret distribution is required.

Checking the init state

GET /api/init is unauthenticated and returns the current state at any time:

curl http://localhost:8000/api/init
# {"state":"dormant","error":null}

Activating the server

Send POST /api/init with the X-Init-API-Key header set to the bootstrap secret. The body is an InitRequest and all fields are optional — only the values you provide override the dormant configuration:

import httpx

client = httpx.Client(base_url="http://localhost:8000")

resp = client.post(
    "/api/init",
    json={
        # Credentials that should not be baked into the warm image arrive here.
        "env": {"LLM_API_KEY": user_api_key},
        # Point at the user's mounted workspace.
        "conversations_path": "/mnt/user-workspace/conversations",
        # Lock down the API to this user's session key.
        "session_api_keys": [user_session_key],
    },
    headers={"X-Init-API-Key": BOOTSTRAP_SECRET_KEY},
)
assert resp.json()["state"] == "ready"

InitRequest fields:

Field	Type	Description
session_api_keys	list[str]	Per-user API keys for subsequent /api/* calls
secret_key	str	Encryption secret (defaults to first session_api_key)
conversations_path	path	Where conversations are persisted
bash_events_dir	path	Where bash events are persisted
env	dict[str, str]	Process env vars set before services start (e.g. credentials)
webhooks	list	Per-user webhooks for event streaming
web_url	str	External server URL for root-path calculation
allow_cors_origins	list[str]	CORS origins added to the localhost allowlist
max_concurrent_runs	int	Override conversation-step concurrency limit

Error handling

If initialization fails, the state rolls back to dormant and the error is stored in GET /api/init response's error field. The orchestrator can then retry POST /api/init:

curl http://localhost:8000/api/init
# {"state":"dormant","error":"ConversationService failed to start: ..."}

Ready-to-run example

Note

This example is available on GitHub: examples/02_remote_agent_server/16_deferred_init.py

This example walks through the full warm-pool lifecycle: starting a dormant server, verifying the 503 gate, activating it via POST /api/init, and running a conversation on the ready server.

<placeholder — auto-synced from agent-sdk>

You can run the example code as-is.

NoteThe model name follows the LiteLLM convention, provider/model_name (for example anthropic/claude-sonnet-4-5-20250929 or openai/gpt-4o). LLM_API_KEY is the API key for the provider you choose.

Bring your own provider key

export LLM_API_KEY="your-api-key"
export LLM_MODEL="anthropic/claude-sonnet-4-5-20250929"  # or openai/gpt-4o, etc.
cd software-agent-sdk
uv run python examples/02_remote_agent_server/16_deferred_init.py

Faheem Code Cloud key

# https://app.faheemcode.ai/settings/api-keys
export LLM_API_KEY="example-user-api-key"
export LLM_MODEL="faheemcode/claude-sonnet-4-5-20250929"
cd software-agent-sdk
uv run python examples/02_remote_agent_server/16_deferred_init.py

Next steps

• Local Agent Server — Run a server in the same process
• Docker Sandbox — Isolated Docker-based deployment
• Settings & Secrets API — Manage per-user secrets securely
• Agent Server Overview — Architecture and deployment options