Pointing an agent at a room

A room is a stateful twin of one system, served at its own URL. Your agent calls it exactly as it would call production. This page covers the hosted MVP console; the self-hosted CLI (rooms record / compile / up / run) uses the same spec, control API and grading.

• Quickstart
• REST API of a room
• Sessions
• Control endpoints
• MCP
• OpenEnv-style adapter
• Room spec
• Scenarios and grading
• CI usage

Quickstart

1. In the console, create a room from the Payroll or HubSpot template, or compile one from a HAR file or OpenAPI spec.
2. Copy its base URL, e.g. https://<host>/rooms/acme-hubspot-1a2b/api.
3. Swap the base URL in your agent's environment. Nothing else changes:

export HUBSPOT_BASE_URL=https://<host>/rooms/acme-hubspot-1a2b/api
python run_agent.py

Then open the room's scenarios, press Run with "External agent", run your agent against the session IDs shown, and press Grade now. Or run a built-in demo agent to see a full report straight away.

REST API of a room

Every resource in the spec gets these endpoints (paths come from the spec, e.g. /crm/v3/objects/contacts):

Unknown endpoints return 404 with the list of modeled resources, so drift shows up loudly.

Sessions

Send X-Room-Session: <id> on every call to get an isolated copy of the room's state. A new session starts from the spec's seed data. Without the header you share the default session. Fifty parallel runs with fifty session IDs never see each other's writes.

Control endpoints

All take the same X-Room-Session header (or "session" in the body / ?session=).

# reset to the spec seed, or to your own seed (resource -> records)
curl -X POST $ROOM/_control/reset -H 'X-Room-Session: t1'
curl -X POST $ROOM/_control/reset -H 'X-Room-Session: t1' -H 'content-type: application/json' \
  -d '{"seed": {"contacts": [{"id": "1", "properties": {"email": "a@b.example"}}]}}'

# read the full state
curl $ROOM/_control/state -H 'X-Room-Session: t1'

# inject faults (latency, error, rate_limit); optional "path" substring filter
curl -X POST $ROOM/_control/inject -H 'X-Room-Session: t1' -d '{"type": "latency", "ms": 1500}'
curl -X POST $ROOM/_control/inject -H 'X-Room-Session: t1' -d '{"type": "error", "status": 500, "rate": 0.3}'
curl -X POST $ROOM/_control/inject -H 'X-Room-Session: t1' -d '{"type": "rate_limit", "limit": 5, "window_s": 10}'
curl -X POST $ROOM/_control/inject -H 'X-Room-Session: t1' -d '{"clear": true}'

# request log
curl $ROOM/_control/requests?session=t1

Here $ROOM is https://<host>/rooms/<room-id> (the base URL without /api). Rate limits answer 429 with Retry-After.

MCP

Every room is also an MCP server at https://<host>/rooms/<room-id>/mcp (streamable HTTP, JSON responses). Tools are generated per resource: list_*, get_*, create_*, update_*, delete_*. MCP calls hit the same state, faults and request log as REST, and honor X-Room-Session.

curl -X POST $ROOM/mcp -H 'content-type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"list_employees","arguments":{"filters":{"department":"Finance"}}}}'

OpenEnv-style adapter (preview)

POST $ROOM/_openenv/reset, POST $ROOM/_openenv/step with {"action": {"method", "path", "body"}}, and GET $ROOM/_openenv/state. The room's Export button downloads the spec, scenarios and an environment manifest. Full OpenEnv packaging is on the roadmap.

Room spec

name: HubSpot CRM (contacts)
base_url_env: HUBSPOT_BASE_URL
resources:
  contacts:
    path: /crm/v3/objects/contacts
    id_field: id
    id_style: string-int      # int | string-int | prefix (id_prefix) | uuid
    id_start: 107
    list_key: results         # list envelope, if the real API uses one
    single_key: null          # item envelope, e.g. ticket
    search: true
    unique: [properties.email]          # or composite: [[employee_id, period]]
    required: []
    timestamps: {created: createdAt, updated: updatedAt}
    seed: [...]
webhooks:                     # only where the real system sends them
  - {on: contacts.created, url: https://your-app.example/hooks/crm}

The compiler produces this from a HAR file (observed endpoints, response bodies as seed, PII redacted) or an OpenAPI spec (collections, envelopes, schema-generated seed). Review it before you rely on it; nested endpoints are listed under notes for manual modeling.

Scenarios and grading

A scenario is a starting state, a trigger, optional faults and assertions on the end state:

{
  "name": "Webinar lead import",
  "instruction": "Upsert these leads; never duplicate; never touch opted-out contacts.",
  "start_state": "seed",
  "trigger": {"type": "instruction", "input": {"leads": [...]}},
  "faults": [{"type": "rate_limit", "limit": 3, "window_s": 1}],
  "assertions": [
    {"type": "count", "resource": "contacts", "where": {"properties.email": "dan@globex.example"}, "equals": 1},
    {"type": "field", "resource": "contacts", "id": "101", "path": "properties.lifecyclestage", "equals": "marketingqualifiedlead"},
    {"type": "field", "resource": "payslips", "where": {"employee_id": "E-1002"}, "path": "gross", "equals": 15495.88, "tolerance": 0.01},
    {"type": "unchanged", "resource": "contacts", "ids": ["102"]},
    {"type": "no_calls", "method": "DELETE"}
  ]
}

Each run creates k isolated sessions (one per trial), resets them to the starting state and injects the faults. Grading diffs the room's state against the assertions. A run passes pass^k only if every trial passes. Reports download as HTML and JUnit XML.

CI usage

The console API accepts Authorization: Bearer $APP_PASSWORD:

# start an external run with 3 trials; returns session ids
curl -X POST https://<host>/app/api/runs -H "Authorization: Bearer $APP_PASSWORD" \
  -H 'content-type: application/json' -d '{"scenario_id": 1, "mode": "external", "trials": 3}'
# ... run your agent once per session id ...
curl -X POST https://<host>/app/api/runs/42/grade -H "Authorization: Bearer $APP_PASSWORD"
curl -o junit.xml https://<host>/app/runs/42/junit.xml -H "Authorization: Bearer $APP_PASSWORD"