Test Plans

Chain several tests into a directed acyclic graph (DAG): each step runs one test under a named browser profile and declares the outputs it records. Every value recorded by a step's ancestors is provided to that step's agent automatically — prompts refer to them in plain language (e.g. "log in with the credentials from the register step"), no template syntax needed. The runner walks the graph in dependency order, runs independent steps in parallel, and shares one logged-in browser across all steps that name the same profile.

The mental model

New to test plans? Read the Test Plans guide first; it explains the concepts in plain language. This page is the API reference: the exact JSON, the addressing rules, and the behaviour you can rely on at run time.

A test plan is a directed acyclic graph (DAG) of steps. Each step runs exactly one test under a named browser profile and declares the outputs it records; every value recorded by a step's ancestors is handed to that step's agent automatically. “Acyclic” means the dependencies can never loop back on themselves. The runner needs a definite order, so a step can never (directly or indirectly) depend on itself.

When you start a run, Aiqaramba freezes the plan into a snapshot and walks the graph. A step is dispatched the moment every step it depends on has finished, so independent branches run in parallel and dependent steps run in order. If a step fails, its descendants are skipped by default (see on_parent_failure).

Addressing, IDs, and errors

Two ways to address a step. In URLs, a step is always its UUID (/test-plans/{id}/steps/{stepId}). Inside a request body, steps reference each other by label: the parents array lists parent labels, and the values an agent receives from its ancestors are namespaced by the producing step's label. Labels are mandatory, unique within a plan, and identifier-shaped (^[A-Za-z_][A-Za-z0-9_]*$, no hyphens).

Step UUIDs are stable across saves as long as the label is unchanged, so a GET → mutate → PUT round-trip preserves them. Renaming a label is treated as delete-old + insert-new.

Validation errors come back as RFC 7807 application/problem+json with per-field violations under the violations extension. Any 422 in this section uses that shape. The validator runs before any data is saved, so a rejected plan never leaves a partial graph behind.

Browser profiles are actors, not steps

This is the single most important design rule, and the one people get wrong first. A profile_name (pattern ^[a-z0-9_-]+$) represents one actor, one person sitting at one browser, not one step.

Every step that names the same profile reuses the same logged-in browser session: the same cookies, local storage, and authenticated identity, carried over in dependency order. Steps with different profiles get fully isolated sessions and may run in parallel.

So the canonical shape is: one small login step per actor, with that actor's remaining steps as descendants on the same profile, each prompt starting from the already-logged-in state. (See the example plan further down, where both steps run as alice in one browser, logged in once.)

Why it matters: giving every step its own profile makes every step log in from scratch. On apps that allow only one active session per user, those parallel logins invalidate each other mid-run. The symptom is steps failing with 500s or redirects back to the login page that a retry cannot fix. Reserve distinct profiles for genuinely different actors (an admin and the teammate they invite, a buyer and a fulfilment user).

What happens when a step fails

Each step carries an on_parent_failure policy controlling what happens when a parent reaches a non-success terminal state:

  • skip (the default) propagates the failure: the step is skipped, and so are its descendants. This is usually what you want, because a checkout step is meaningless if signup never completed.
  • run runs anyway once the parents are terminal. Use this for cleanup, teardown, or verification that does not actually depend on the parent succeeding.

Moving data between steps

There are two sources of data a step can use: plan-level variables (template-substituted into the prompt) and ancestor step outputs (handed to the agent automatically — no template syntax).

1. Plan-level variables

Declared on the plan as a top-level variables object: name → {description, type, default?}. A variable without a default is required, and the caller must supply it when starting a run via the variables body of POST /api/v1/test-plans/{id}/runs. Missing required variables are rejected with HTTP 422.

A test can directly reference a plan variable inside its instructions using {{ plan.variables.<name> }}. The runner resolves the reference to a plain string before the agent sees the prompt.

2. Step outputs

Each step declares the data its test will publish for downstream steps as name → {description, type?, format?}. The test is responsible for actually emitting these values during its run.

Every value recorded by a step's ancestors is provided to that step's agent automatically, namespaced by the producing step's label and carrying the declared description. Prompts refer to them in plain language — “log in with the credentials recorded by the register step”. Plan variables are the only values substituted with template syntax.

Worked example

Plan variables:

{ "plan_name": {"type": "string", "default": "Acme Inc."} }

Step signup declares outputs:

{ "user_id": {"description": "ID of the new user", "type": "string", "format": "uuid"} }

The onboarding test's prompt_template (running as a step dependent on signup) injects the plan variable with template syntax and refers to the signup step's output in plain language:

Welcome to {{ plan.variables.plan_name }}! Verify the profile page shows the user id recorded by the signup step.

See the Tests reference for the full prompt-template data model.

Writing prompts for steps

A step runs an ordinary test, but the test's prompt is read in the context of a plan. A few rules keep that context from biting you:

  • Standalone test template variables are NOT rendered in plan runs. A test run on its own can use {{.EntryURL}}, {{.RunIndex}}, and {{.Vars.*}}. In a plan, those tokens are left untouched and the agent sees raw braces. Use literal URLs, or {{ plan.variables.<name> }} to inject values supplied at run time.
  • Descendant prompts assume the shared session set up by an earlier step on the same profile, but should carry a credential fallback in case of an unexpected logout (“when not signed in, log in as …”).
  • End every step with an explicit verification naming an observable outcome, and keep plans to roughly eight steps. Split bigger pipelines into separate phases rather than one sprawling graph.

Common pitfalls

  • A separate profile per step. The most common mistake; see Browser profiles are actors. Give each actor one profile and share it across that actor's steps; a fresh profile per step makes every step log in again.
  • Required plan variable not supplied at run time. POST /api/v1/test-plans/{id}/runs returns HTTP 422 with the missing-variable name in the response.
GET /api/v1/test-plan-runs/{id}

Get a test plan run

Parameters

ParameterTypeInRequiredDescription
iduuidpathYesTest plan run ID.

Example Request

curl
curl "http://app.aiqaramba.com/api/v1/test-plan-runs/<id>" \
  -H "Authorization: Bearer $AIQA_API_KEY"

Status Codes

CodeDescription
200Run with node runs.
401Authentication is missing or invalid.
404The requested resource does not exist.
500The server could not complete the request.

Response Fields (200)

FieldTypeDescription
iduuid
plan_iduuid
tenant_iduuid
project_iduuid
statusstring
stoppingboolean
graph_snapshotobject
variablesobject
tunnel_urluri
node_runsobject[]
node_runs[].iduuid
node_runs[].node_iduuid
node_runs[].agent_iduuid
node_runs[].statusstring
node_runs[].attemptinteger
node_runs[].outputsobject
node_runs[].started_atdate-time
node_runs[].finished_atdate-time
node_runs[].created_atdate-time
started_atdate-time
finished_atdate-time
created_atdate-time
POST /api/v1/test-plan-runs/{id}/stop

Stop a test plan run

Parameters

ParameterTypeInRequiredDescription
iduuidpathYesTest plan run ID.

Example Request

curl
curl -X POST "http://app.aiqaramba.com/api/v1/test-plan-runs/<id>/stop" \
  -H "Authorization: Bearer $AIQA_API_KEY"

Status Codes

CodeDescription
202Stop accepted.
401Authentication is missing or invalid.
404The requested resource does not exist.
500The server could not complete the request.

Response Fields (202)

FieldTypeDescription
iduuid
plan_iduuid
tenant_iduuid
project_iduuid
statusstring
stoppingboolean
graph_snapshotobject
variablesobject
tunnel_urluri
node_runsobject[]
node_runs[].iduuid
node_runs[].node_iduuid
node_runs[].agent_iduuid
node_runs[].statusstring
node_runs[].attemptinteger
node_runs[].outputsobject
node_runs[].started_atdate-time
node_runs[].finished_atdate-time
node_runs[].created_atdate-time
started_atdate-time
finished_atdate-time
created_atdate-time
GET /api/v1/test-plans

List test plans

Parameters

ParameterTypeInRequiredDescription
project_iduuidqueryYesOwning project ID.

Example Request

curl
curl "http://app.aiqaramba.com/api/v1/test-plans?project_id=<project_id>" \
  -H "Authorization: Bearer $AIQA_API_KEY"

Status Codes

CodeDescription
200Project plan summaries.
400The request is malformed or failed validation.
401Authentication is missing or invalid.
500The server could not complete the request.

Response Fields (200)

FieldTypeDescription
test_plansobject[]
test_plans[].iduuid
test_plans[].tenant_iduuid
test_plans[].project_iduuid
test_plans[].namestring
test_plans[].descriptionstring
test_plans[].variablesobject
test_plans[].step_countinteger
test_plans[].created_atdate-time
test_plans[].updated_atdate-time
POST /api/v1/test-plans

Create a test plan

Request Body (application/json)

FieldTypeRequiredDescription
project_iduuidYesProject that owns the plan. Every referenced test must belong to the same project.
namestringYesPlan name (required, ≤120 chars).
descriptionstringNoOptional human description (≤500 chars).
variablesjsonNoPlan-level variables the caller will supply at run time. Map of name → {description, type, default?}. Required variables are those without a default.
stepsjsonNoInitial graph. May be empty; use POST /test-plans/{id}/steps to add steps incrementally.

Example Request

curl
curl -X POST "http://app.aiqaramba.com/api/v1/test-plans" \
  -H "Authorization: Bearer $AIQA_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "project_id": "<uuid>",
    "name": "<string>"
  }'

Status Codes

CodeDescription
201Plan created.
400The request is malformed or failed validation.
401Authentication is missing or invalid.
422The request is structurally valid but cannot be processed.
500The server could not complete the request.

Response Fields (201)

FieldTypeDescription
iduuid
tenant_iduuid
project_iduuid
namestring
descriptionstring
variablesobject
stepsany[]
created_atdate-time
updated_atdate-time
GET /api/v1/test-plans/{id}

Get a hydrated test plan

Parameters

ParameterTypeInRequiredDescription
iduuidpathYesTest plan ID.

Example Request

curl
curl "http://app.aiqaramba.com/api/v1/test-plans/<id>" \
  -H "Authorization: Bearer $AIQA_API_KEY"

Status Codes

CodeDescription
200Hydrated plan.
401Authentication is missing or invalid.
404The requested resource does not exist.
500The server could not complete the request.

Response Fields (200)

FieldTypeDescription
iduuid
tenant_iduuid
project_iduuid
namestring
descriptionstring
variablesobject
stepsany[]
created_atdate-time
updated_atdate-time
PUT /api/v1/test-plans/{id}

Replace a test plan

Parameters

ParameterTypeInRequiredDescription
iduuidpathYesTest plan ID.

Request Body (application/json)

FieldTypeRequiredDescription
namestringYesPlan name (required, ≤120 chars).
descriptionstringNoOptional human description (≤500 chars).
variablesjsonNoPlan-level variables the caller will supply at run time. Map of name → {description, type, default?}. Required variables are those without a default.
stepsjsonNoInitial graph. May be empty; use POST /test-plans/{id}/steps to add steps incrementally.

Example Request

curl
curl -X PUT "http://app.aiqaramba.com/api/v1/test-plans/<id>" \
  -H "Authorization: Bearer $AIQA_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "<string>"
  }'

Status Codes

CodeDescription
200Replaced plan.
401Authentication is missing or invalid.
404The requested resource does not exist.
422The request is structurally valid but cannot be processed.
500The server could not complete the request.

Response Fields (200)

FieldTypeDescription
iduuid
tenant_iduuid
project_iduuid
namestring
descriptionstring
variablesobject
stepsany[]
created_atdate-time
updated_atdate-time
DELETE /api/v1/test-plans/{id}

Delete a test plan

Parameters

ParameterTypeInRequiredDescription
iduuidpathYesTest plan ID.

Example Request

curl
curl -X DELETE "http://app.aiqaramba.com/api/v1/test-plans/<id>" \
  -H "Authorization: Bearer $AIQA_API_KEY"

Status Codes

CodeDescription
204Plan deleted.
401Authentication is missing or invalid.
404The requested resource does not exist.
500The server could not complete the request.
GET /api/v1/test-plans/{id}/runs

List test plan runs

Parameters

ParameterTypeInRequiredDescription
iduuidpathYesTest plan ID.

Example Request

curl
curl "http://app.aiqaramba.com/api/v1/test-plans/<id>/runs" \
  -H "Authorization: Bearer $AIQA_API_KEY"

Status Codes

CodeDescription
200Plan runs.
401Authentication is missing or invalid.
404The requested resource does not exist.
500The server could not complete the request.

Response Fields (200)

FieldTypeDescription
runsobject[]
runs[].iduuid
runs[].plan_iduuid
runs[].tenant_iduuid
runs[].project_iduuid
runs[].statusstring
runs[].stoppingboolean
runs[].graph_snapshotobject
runs[].variablesobject
runs[].tunnel_urluri
runs[].node_runsobject[]
runs[].node_runs[].iduuid
runs[].node_runs[].node_iduuid
runs[].node_runs[].agent_iduuid
runs[].node_runs[].statusstring
runs[].node_runs[].attemptinteger
runs[].node_runs[].outputsobject
runs[].node_runs[].started_atdate-time
runs[].node_runs[].finished_atdate-time
runs[].node_runs[].created_atdate-time
runs[].started_atdate-time
runs[].finished_atdate-time
runs[].created_atdate-time
POST /api/v1/test-plans/{id}/runs

Start a test plan run

Parameters

ParameterTypeInRequiredDescription
iduuidpathYesTest plan ID.

Request Body (application/json)

FieldTypeRequiredDescription
variablesjsonNoCaller-supplied values for the plan's declared variables. Unknown variable names are rejected.
tunnelstringNoHostname of a live tunnel owned by this tenant. Every agent in the run starts at its test's entry URL rewritten onto this host, and is instructed to substitute this host for any other URL its prompt mentions.

Example Request

curl
curl -X POST "http://app.aiqaramba.com/api/v1/test-plans/<id>/runs" \
  -H "Authorization: Bearer $AIQA_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "variables": {},
    "tunnel": "<string>"
  }'

Status Codes

CodeDescription
201Run started.
401Authentication is missing or invalid.
404The requested resource does not exist.
422The request is structurally valid but cannot be processed.
500The server could not complete the request.
503The requested feature is not configured on this server.

Response Fields (201)

FieldTypeDescription
iduuid
plan_iduuid
tenant_iduuid
project_iduuid
statusstring
stoppingboolean
graph_snapshotobject
variablesobject
tunnel_urluri
node_runsobject[]
node_runs[].iduuid
node_runs[].node_iduuid
node_runs[].agent_iduuid
node_runs[].statusstring
node_runs[].attemptinteger
node_runs[].outputsobject
node_runs[].started_atdate-time
node_runs[].finished_atdate-time
node_runs[].created_atdate-time
started_atdate-time
finished_atdate-time
created_atdate-time
POST /api/v1/test-plans/{id}/steps

Add a plan step

Parameters

ParameterTypeInRequiredDescription
iduuidpathYesTest plan ID.

Request Body (application/json)

FieldTypeRequiredDescription
labelstringYesIdentifier-shaped label (^[A-Za-z_][A-Za-z0-9_]*$), unique within the plan. Descendant steps see this step's recorded outputs namespaced under this label, and prompts refer to the step by it.
test_iduuidYesTest to run for this step. Must belong to the plan's project.
profile_namestringYesBrowser identity for the step (^[a-z0-9_-]+$). One profile per ACTOR, shared across that actor's steps: steps sharing a profile_name reuse the same logged-in browser within a run (serialised, state carries over), so log in once in an early step and let descendants start from the logged-in state. Distinct profiles are separate sessions that may run in parallel — only use them for genuinely different actors.
on_parent_failurestringNoWhat to do when a parent step reaches a non-success terminal state. One of skip (default — propagate failure to descendants) or run (run regardless once parents are terminal).
parentsstring[]NoSibling step labels this step depends on. Empty array for entry-point steps.
outputsjsonNoMap of output name → {description (required), type?, format?}. Names must match the label pattern. type ∈ {string, number, boolean, object}; format ∈ {uuid, url, email}. Values recorded here are automatically provided to every descendant step's agent.
role_iduuidNoOptional role to attach to the dispatched agent.

Example Request

curl
curl -X POST "http://app.aiqaramba.com/api/v1/test-plans/<id>/steps" \
  -H "Authorization: Bearer $AIQA_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "label": "<string>",
    "test_id": "<uuid>",
    "profile_name": "<string>"
  }'

Status Codes

CodeDescription
201Updated hydrated plan.
401Authentication is missing or invalid.
404The requested resource does not exist.
422The request is structurally valid but cannot be processed.
500The server could not complete the request.

Response Fields (201)

FieldTypeDescription
iduuid
tenant_iduuid
project_iduuid
namestring
descriptionstring
variablesobject
stepsany[]
created_atdate-time
updated_atdate-time
DELETE /api/v1/test-plans/{id}/steps/{stepId}

Delete a plan step

Parameters

ParameterTypeInRequiredDescription
iduuidpathYesTest plan ID.
stepIduuidpathYesStable plan step ID.

Example Request

curl
curl -X DELETE "http://app.aiqaramba.com/api/v1/test-plans/<id>/steps/<stepId>" \
  -H "Authorization: Bearer $AIQA_API_KEY"

Status Codes

CodeDescription
200Plan after step removal.
401Authentication is missing or invalid.
404The requested resource does not exist.
500The server could not complete the request.

Response Fields (200)

FieldTypeDescription
iduuid
tenant_iduuid
project_iduuid
namestring
descriptionstring
variablesobject
stepsany[]
created_atdate-time
updated_atdate-time
PATCH /api/v1/test-plans/{id}/steps/{stepId}

Replace a plan step

Parameters

ParameterTypeInRequiredDescription
iduuidpathYesTest plan ID.
stepIduuidpathYesStable plan step ID.

Request Body (application/json)

FieldTypeRequiredDescription
labelstringYesIdentifier-shaped label (^[A-Za-z_][A-Za-z0-9_]*$), unique within the plan. Descendant steps see this step's recorded outputs namespaced under this label, and prompts refer to the step by it.
test_iduuidYesTest to run for this step. Must belong to the plan's project.
profile_namestringYesBrowser identity for the step (^[a-z0-9_-]+$). One profile per ACTOR, shared across that actor's steps: steps sharing a profile_name reuse the same logged-in browser within a run (serialised, state carries over), so log in once in an early step and let descendants start from the logged-in state. Distinct profiles are separate sessions that may run in parallel — only use them for genuinely different actors.
on_parent_failurestringNoWhat to do when a parent step reaches a non-success terminal state. One of skip (default — propagate failure to descendants) or run (run regardless once parents are terminal).
parentsstring[]NoSibling step labels this step depends on. Empty array for entry-point steps.
outputsjsonNoMap of output name → {description (required), type?, format?}. Names must match the label pattern. type ∈ {string, number, boolean, object}; format ∈ {uuid, url, email}. Values recorded here are automatically provided to every descendant step's agent.
role_iduuidNoOptional role to attach to the dispatched agent.

Example Request

curl
curl -X PATCH "http://app.aiqaramba.com/api/v1/test-plans/<id>/steps/<stepId>" \
  -H "Authorization: Bearer $AIQA_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "label": "<string>",
    "test_id": "<uuid>",
    "profile_name": "<string>"
  }'

Status Codes

CodeDescription
200Plan with replaced step.
401Authentication is missing or invalid.
404The requested resource does not exist.
422The request is structurally valid but cannot be processed.
500The server could not complete the request.

Response Fields (200)

FieldTypeDescription
iduuid
tenant_iduuid
project_iduuid
namestring
descriptionstring
variablesobject
stepsany[]
created_atdate-time
updated_atdate-time