AI Testing in PracticePractical guides for experienced software testers moving into AI/LLM testing — mutation testing, prompt contracts, evals, and CI gates. By the author of the book “AI Testing in Practice”.https://aitestinginpractice.com/Flaky AI tests are usually bad tests: how to stabilize a non-deterministic suitehttps://aitestinginpractice.com/articles/flaky-ai-tests-bad-tests-stabilize/https://aitestinginpractice.com/articles/flaky-ai-tests-bad-tests-stabilize/Most AI test flakiness isn't the model's fault — it's exact-match assertions on output that was never guaranteed to stay the same. Fix the assertion, not the AI.Sun, 01 Nov 2026 00:00:00 GMTPrompt injection is a QA problem now — here are the tests you should be writinghttps://aitestinginpractice.com/articles/prompt-injection-qa-tests/https://aitestinginpractice.com/articles/prompt-injection-qa-tests/Prompt injection attacks hijack LLM behavior through malicious user input or retrieved documents. These are the four test categories every AI application needs.Sat, 31 Oct 2026 00:00:00 GMTHow to catch hallucinations in tests without a human reading every answerhttps://aitestinginpractice.com/articles/catch-hallucinations-in-tests/https://aitestinginpractice.com/articles/catch-hallucinations-in-tests/Grounding checks, NLI contradiction detection, and a narrow LLM-as-judge rubric automate hallucination detection — no human in the loop required.Fri, 30 Oct 2026 00:00:00 GMTCodex or Claude wrote my tests — how do I actually trust them?https://aitestinginpractice.com/articles/trust-ai-generated-tests/https://aitestinginpractice.com/articles/trust-ai-generated-tests/A green AI-generated suite proves the code does what it already does, bugs and all. The only way to earn confidence is to break the code on purpose and watch the tests fail.Fri, 30 Oct 2026 00:00:00 GMT90% coverage and the bug still shipped: fixing your signal-to-noisehttps://aitestinginpractice.com/articles/coverage-signal-to-noise-bug-shipped/https://aitestinginpractice.com/articles/coverage-signal-to-noise-bug-shipped/Coverage counts line executions, not correctness. A 90% score built on weak assertions is noise. Here's how to measure signal instead.Thu, 29 Oct 2026 00:00:00 GMTFrom experienced tester to AI testing: the roadmap, in orderhttps://aitestinginpractice.com/articles/experienced-tester-ai-testing-roadmap/https://aitestinginpractice.com/articles/experienced-tester-ai-testing-roadmap/You don't need a data-science degree to test AI. Here's the order to actually learn it, built on what you already know as a tester.Thu, 29 Oct 2026 00:00:00 GMTHow to test an LLM or RAG app when the same prompt gives different valid answershttps://aitestinginpractice.com/articles/testing-llm-rag-nondeterministic-outputs/https://aitestinginpractice.com/articles/testing-llm-rag-nondeterministic-outputs/Use property checks, scored evals, and a golden dataset — never exact-match assertions. Non-determinism isn't the problem; asserting the wrong thing is.Thu, 29 Oct 2026 00:00:00 GMTReal bug or model failure? How to tell the difference in AI testshttps://aitestinginpractice.com/articles/real-bug-or-model-failure/https://aitestinginpractice.com/articles/real-bug-or-model-failure/When an AI test goes red, it's either a genuine defect or the model behaving differently. Here's a practical way to separate the two instead of guessing.Tue, 27 Oct 2026 00:00:00 GMT