Writing Effective Tests

An agent is only as good as the instructions it receives. The whole point of testing with Aiqaramba is that those instructions no longer have to be exact. A Playwright script needs precise code that clicks this selector, then that one, in this order. Our agents need none of that. They read the page, understand your intent, and figure out the clicks themselves.

In fact, a little vagueness helps. "Submit the form and check it worked" ages far better than "click the button with id #submit-btn," because the agent adapts when your UI changes while the rigid instruction breaks. This page collects the practices we have found produce reliable, repeatable tests.

What goes into a prompt

Agents cope well with vagueness, but they cannot read your mind. Every good prompt gives the agent three things:

What to verify. The specific thing you consider "working." This is the part people most often leave out.
The information it needs. URLs, credentials, which account to use. Anything the agent cannot invent on its own.
A rough outline of the approach. A loose sketch of the steps, not a click-by-click script.

The first two matter most. The days of naming exact buttons are over, but you still have to say what you want validated. Leave it out and the agent will still "work" . It just invents its own best-effort opinion of success, which makes your results unpredictable and lets obvious bugs slip through.

Compare these two prompts for the same login page:

Underspecified, so the agent guesses what "working" means:

Go to https://app.example.com and log in with these credentials:

Username: bob@example.com
Password: Test123!

Better, because the agent knows exactly what to check:

Go to https://app.example.com and verify whether you can log in with:

Username: bob@example.com
Password: Test123!

Verify that:
- Incorrect credentials return nicely rendered, useful error messages.
- Weird unicode characters do not break validation.
- Empty values produce a clear, non-breaking error.

The second prompt constrains the agent to the checks you care about and only reports success when they all pass. Notice that the checks can still be vague in wording, like "weird unicode characters" or "empty values." You just have to mention them. The first prompt runs fine but rarely tries odd inputs on its own, so it quietly misses the very things you wanted to catch.

Give the agent what it needs

Agents are go-getters that proactively push to complete the task. That is usually a strength, but it turns into odd behaviour when you have left out something critical. The usual culprits are missing or wrong credentials and missing URLs.

A prompt like this will not get you far:

Test my login

Which page? Which part of the login? With what credentials? Not even the best human tester could act on that. You can be vague, but not empty. Always hand over the information the task genuinely requires. Think of it exactly like briefing a colleague: the agent knows what "log in" and "go to" mean, but it cannot supply the URL and the test account for you.

Testing email flows

Email has always been a friction point for automation. Magic links, one-time codes, and signup confirmations all live in an inbox the test cannot normally reach. Aiqaramba solves this with mailboxes. A mailbox is a real email address your agents can read: create one in the dashboard or via the API, pick a name, and every message sent to that address (for example qa-signups@agents.aiqaramba.com) is stored against it. Attach the mailbox to a test and its agents can read what lands there.

An agent knows which mailboxes are attached to it and will reach for one whenever a flow needs an email. That convenience has a sharp edge: attach a mailbox, ask the agent to "sign up," and it will happily use the mailbox address as the account email even if that was not your intent. So be explicit. Tell it to "use the attached mailbox to do X" and you are never surprised.

Sign up for a new account at https://app.example.com/signup.
Use the attached mailbox (qa-signups@agents.aiqaramba.com) as the email
address. After submitting, read the inbox of that mailbox, open the
confirmation email, and follow the verification link.

Verify that the account is confirmed and you land on the dashboard.

Handling credentials securely

For test credentials, we strongly recommend using accounts tied to mailboxes you created in Aiqaramba. It makes the whole workflow smoother, and it keeps you clear of real user data.

Notice what we do not offer: a vault to "securely" store and hand secrets to agents. That is deliberate, and it is not a gap we plan to fill. There is no way to give an LLM a credential in a form it cannot leak. At some point the agent needs the plain-text value to type it into a login form, and the moment it holds that value, in a non-deterministic system, it can surface anywhere. Encrypted storage in front of that reality is security theatre. We would rather tell you the truth than sell you the illusion of safety.

The real fix is simpler: do not hand over anything sensitive in the first place. Use throwaway test accounts, point at staging rather than production, use fake credit-card numbers. You cannot leak what you never provided, and that is good practice regardless of Aiqaramba.