Association for Software Testing

Evaluating AI agents as black-box systems fails to catch improper tool invocations and schema violations. By implementing layered isolation and validating JSON payloads before execution, QA engineers can verify agent decision-making safely. Injecting mock error responses into the agent observation window ensures robust error handling without exposing live staging environments...

When
Mireia Cano
and I received the confirmation that our paired talk “Security Cham...

This episode explores the emerging methods for safely evaluating autonomous AI agents and mitigating test flakiness from modern model architectures. We review the decoupled architecture of AgentCompass for running parallelized agent evaluations, alongside Copy-on-Write Scoring for isolating database writes via PostgreSQL triggers. Additionally, we discuss programmatic golden ...

AI coding agents are increasing development velocity, causing traditional test scripts bound to DOM structures to break frequently. Transitioning to intent-based adversarial validation allows teams to deploy autonomous agents that navigate applications visually. This decoupling of business logic from structural elements eliminates the maintenance overhead associated with fixi...

I’m taking a break for a bit from AI social commentary and the increasingly tedious (and boring) work of watching the AI Confidence Bros and test tool vendors discover testing for the first time. I’m going to (try) to focus on some podcasts and AI integration e.g. what happens when these systems run into organizations, people, controls, incentives, legacy technology and reali...