Self-healing tests and black-box audits: how our QA agent earns trust
Generated tests that break on every refactor are worse than no tests. Our QA Engineer agent generates suites from requirements, heals them through change, and audits the running app like a real tester.
In an AI-built application, QA carries a special burden: it is the stage that decides whether anything the other agents claimed is actually true. That's why our QA Engineer agent doesn't just run the developer's tests — it independently generates its own, and then goes further and attacks the running application from the outside.
Tests come from requirements, not from code
Tests generated by reading the implementation inherit the implementation's blind spots — if the code forgot an edge case, so does the test. Our QA agent generates its suites from the PRD and the API contracts: what the product is supposed to do, for whom, under what conditions. When a requirement has no passing test, that's a finding, whether or not the code 'looks right'.
Self-healing, because software keeps moving
The classic failure of test automation is brittleness: a renamed button or a refactored component and half the suite goes red for reasons that have nothing to do with behavior. In an agentic pipeline, where the developer agent iterates in fast vertical slices, brittle tests would drown every real signal in noise.
So the QA agent's tests are self-healing: when a test fails, the agent first determines whether behavior changed or only structure did. Structural drift — a selector, a layout change, a renamed field with identical semantics — is repaired automatically and noted. Behavioral change fails loudly and goes back through MAC to a human gate. The suite stays green for the right reasons and red for the right reasons.
The black-box audit: testing like a user, not like an author
Finally, the QA agent runs a live black-box audit against the deployed application — no source code access, just the app, the way a real tester (or a real user having a bad day) would use it. Broken flows, dead ends, error states, inputs nobody planned for.
The audit's findings flow back into MAC as work items with evidence attached, prioritized into the backlog by the product manager agent, and fixed by the developer agent in the next slice — the same loop a healthy human team runs, executed continuously.
Trust in an AI software team isn't something we can ask for. It has to be produced, stage by stage, by agents whose job is to doubt each other's work. QA is where that doubt pays off.