QA vs QE: Engineering Quality Into Non-Deterministic AI — feat. Janna Loeffler (Quality Engineering Leader)
JANNA LOEFFLER — Janna Loeffler — Quality Engineering Leader
When your system gives a different answer every time, "assuring" quality is a trap — you have to engineer it in. Janna Loeffler joins Alex and Kuba to unpack why the shift from qua…
Show notes
When your system gives a different answer every time, "assuring" quality is a trap — you have to engineer it in. Janna Loeffler joins Alex and Kuba to unpack why the shift from quality assurance to quality engineering (and even "quality enablement") matters more than ever in the age of LLMs and agents. She argues that AI didn't kill the tester's job — it made human insight, risk judgement and context-driven thinking essential, because non-deterministic output turns testing from clean pass/fail into questions of accuracy, relevance and intent. Janna makes the case for an "army" of single-purpose agents that handle the mundane — regression, accessibility, OWASP Top 10 security checks — so testers can focus on outcomes over outputs. The conversation digs into the real danger of eroding guardrails, including a chilling example of a system talked past its safety limits within minutes, and why model providers like Anthropic and OpenAI are pushing safety responsibility down onto the customers who deploy them. Along the way she shares how she vibe-coded a unit-test-writing agent in Cursor in a single day, why diverse test teams are the antidote to biased models, and why the emerging "AI ar