How to Test AI Features When the Output Is Never the Same Twice

Ask the same question to the same model twice and you can get two different, both correct, answers. A test written the traditional way, one input mapped to one expected output, fails against the second answer even though nothing is actually broken.

That's not a testing problem you fix by pinning the temperature to zero. It's a sign the assertion itself is the wrong tool for this layer of the system.

Separate what's deterministic from what isn't

Not everything in an AI feature is non-deterministic, and treating all of it that way throws away your easiest wins.

The scaffolding around the model, the API contract, the tool-call schema, authentication, rate limiting, error handling when the model call fails, should get regular, exact-match tests. If a tool call comes back with a field missing or the wrong type, that's a hard failure, not a judgment call. Write those the way you'd test any other API. Our API testing practices apply here without modification.

The generative core, the actual text or decision the model produces, is where you need a different approach entirely.

What to test instead of exact output

Structural checks first. Before asking whether the content is good, ask whether the response is even shaped correctly: valid JSON if JSON was requested, the right fields present, the tool the model was supposed to call actually called. These are still deterministic and still cheap. Most bugs in production AI features are structural failures caught here, not subtle content problems.

Golden sets with acceptable ranges, not single expected strings. Build a fixed set of representative inputs and, for each one, define what a passing response has to contain or avoid, not what it has to say word for word. "Must mention the refund policy" is testable. "Must say exactly this sentence" is not.

LLM-as-judge, used carefully. A second model can score a response against a rubric (accuracy, tone, whether it followed instructions) more cheaply than a human reviewing every regression run. It's also its own source of noise: a judge model has its own failure modes and its own cost. Treat judge scores as a percentage pass threshold across the golden set, not a per-case pass/fail gate, and don't expect 100%. A model that hits 92% on a stable rubric and holds that number release over release is a suite doing its job.

Regression testing for prompts

Prompt changes need the same discipline as code changes: run the new prompt against the full golden set before it ships, and compare the pass rate to the baseline. A prompt tweak that improves one example and quietly breaks twelve others is the AI-feature equivalent of a regression, and it's invisible without a fixed set to check against.

What this doesn't solve

None of this catches a model that's confidently wrong about something your golden set didn't anticipate. Eval-based testing is only as good as the scenarios you thought to include, which is exactly why exploratory testing still matters even more here than in a deterministic UI: someone has to go looking for the input nobody wrote a golden case for.