Regression Testing

Automatically determine which test journeys to run based on code changes. The regression endpoint analyzes commit messages, PR descriptions, and file diffs against your project's app map and journey catalog using an LLM, then triggers the matched journeys as regression tests. Designed for CI/CD integration — call this after every deploy to run only the tests that matter. At most 5 journeys are triggered per call; if the LLM matches more, only the highest-ranked five run. **When no journey matches** the changes (or the matcher flags a coverage gap), the endpoint also tries to *auto-generate* a fresh journey from the commit context and either the project's app map (if a discovery has completed) or the project's entry URL (if no discovery exists yet — discovery is expensive and most teams won't run it on every release cadence). The generated journey is exploratory in the no-map case but better than nothing for the "we shipped a new surface and there's no journey for it" case. The response's `auto_journey_status` field surfaces what the auto-generation step did so CI can render a meaningful summary instead of an empty result.

CI/CD Setup

Add regression tests to your deploy pipeline using the official GitHub Action. After each deploy, the action sends your recent commits to Aiqaramba, which uses an LLM to match changes against your project's journeys and triggers the relevant tests.

GitHub Actions

Add this step to your deploy workflow (after the deploy step):

- uses: alex-ai-eu/aiqaramba/action@main with: api-key: ${{ secrets.AIQARAMBA_API_KEY }} project-id: ${{ vars.AIQARAMBA_PROJECT_ID }}

Set these in your repo's Settings → Secrets and variables → Actions:

  • AIQARAMBA_API_KEY (secret) — your API key from Account → API Keys
  • AIQARAMBA_PROJECT_ID (variable) — UUID of the project to test against

Options

  • api-base — API URL (default: https://app.aiqaramba.com)
  • fetch-depth — how many commits to analyze (default: 5)
  • dry-run — set to true to preview matches without running agents

Outputs

The action sets these outputs for subsequent steps:

  • run-id — regression run ID (track results in the Regressions page)
  • matched — number of journeys matched to your changes
  • triggered — number of agents started

Other CI systems

For non-GitHub CI, call the API directly with curl:

curl -X POST \ -H "Authorization: Bearer $AIQARAMBA_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "commits": [{"message": "your commit msg", "files_changed": ["src/app.ts"]}], "commit_sha": "'$CI_COMMIT_SHA'" }' \ https://app.aiqaramba.com/api/v1/projects/$PROJECT_ID/regression

Prerequisites

  • Journeys — your project needs test journeys defined. Run a Discovery first to generate suggested journeys, then customize them.
  • App map — a completed discovery gives the LLM context about your app's pages and routes, improving journey matching accuracy.
  • Mailboxes — for flows that need magic links, OTP codes, or any email-based verification, create a mailbox and attach it to the relevant journey so the agent can read inbound mail.

Version Tracking

When you pass commit_sha and version, each regression run is tagged with the deployed version. This appears on the Regressions page so you can see exactly which version each test run covers. The GitHub Action does this automatically.

POST /api/v1/projects/{id}/regression

Trigger regression tests

Analyzes code changes against the project's journeys and app map. Uses an LLM to match changes to relevant test flows, then runs the matched journeys. At least one of commits, pull_requests, pr_title, or diff_summary is required. Use dry_run to preview which journeys would be triggered without running them.

Parameters

ParameterTypeInRequiredDescription
iduuidpathYesProject ID
commitsarraybodyNoList of commits with messages and changed files
pull_requestsarraybodyNoList of pull requests included in this deploy. Each entry: number, title, body, labels[]. Supersedes the legacy flat pr_title / pr_body fields when both are sent.
diff_summarystringbodyNoHigh-level summary of all changes (e.g. git diff --stat output)
pr_titlestringbodyNoPull request title. Legacy — prefer the structured pull_requests array for new integrations.
pr_bodystringbodyNoPull request description. Legacy — prefer the structured pull_requests array for new integrations.
dry_runbooleanbodyNoIf true, return matched journeys without running them (default: false)
commit_shastringbodyNoGit commit SHA of the deploy being tested
versionstringbodyNoVersion tag or git describe output of the deploy. Echoed back in the response for run-tracking.
environmentstringbodyNoEnvironment being tested (e.g. 'production', 'staging'). Echoed in the response.
target_urlstringbodyNoURL of the deployed application being tested. Echoed in the response.
skipped_countintegerbodyNoNumber of commits filtered out by skip tokens (e.g. [skip aiq]). Informational — echoed in the response. (default: 0)

Status Codes

CodeDescription
200Regression run started (or dry-run results returned)
400Validation error — at least one of commits, pull_requests, pr_title, or diff_summary required
401Unauthorized
404Project not found

Response Body

{
  "regression_run_id": "550e8400-e29b-41d4-a716-446655440000",
  "project_id": "660e8400-e29b-41d4-a716-446655440000",
  "matched_journeys": [
    {
      "id": "770e8400-e29b-41d4-a716-446655440000",
      "name": "Checkout Flow",
      "reason": "Commit modifies checkout validation logic"
    }
  ],
  "suggested_journeys": [
    {
      "id": "880e8400-e29b-41d4-a716-446655440000",
      "name": "Refund Flow"
    }
  ],
  "coverage_gap": {
    "files": ["src/refunds.ts"],
    "reason": "No existing journey exercises the refund path"
  },
  "agents": [
    {
      "id": "550e8400-e29b-41d4-a716-446655440000",
      "name": "Checkout Flow",
      "journey_id": "770e8400-e29b-41d4-a716-446655440000"
    }
  ],
  "dry_run": false,
  "version": "v0.1.2-3-g7050747",
  "environment": "production",
  "target_url": "https://app.example.com",
  "auto_journey_status": "generated_and_run",
  "warnings": ["Latest discovery is 12 days old (completed 2025-01-03). Consider re-running discovery for accurate regression matching."]
}
POST /api/v1/projects/{id}/regression
cURL 
Response