Here’s a slightly uncomfortable fact about most “AI features.” The model call itself is usually one line. Everything else, the part that decides what goes into the prompt, what happens when the answer comes back broken, and what the user sees when the model is slow or down, is ordinary code written by ordinary developers. Often quickly, under deadline, and tested mostly by trying a few prompts and eyeballing the results.
That surrounding code is where a lot of real-world AI failures actually live. Not the model being wrong, but the app handling a strange model response badly. White box testing is a good fit for exactly this problem, because it doesn’t care about the model’s intelligence. It cares about which paths through your code actually ran.
The model is a small part of the feature
Take a fairly typical AI-powered support assistant. Before the model is ever called, code decides which documents to include, trims the conversation so it fits the context window, and fills in a prompt template. After the call, more code parses the answer, checks it against an expected structure, filters anything that shouldn’t reach a user, and maybe triggers a follow-up action like creating a ticket.
Each of those steps has branches. What if retrieval finds nothing? What if the conversation is too long? What if the answer isn’t valid JSON? What if a guardrail flags it? Every one of those branches is code someone wrote, and every one of them can be wrong in ways the happy-path demo never shows.
Where the risky branches hide
Some branches around an LLM call almost never run during casual testing, and they tend to be the important ones.
Parsing failures are a classic. Even with structured output modes, a model can sometimes return something that doesn’t match what the code expects: a missing field, an extra sentence before the JSON, a truncated response. The branch that handles that case is often written once and never exercised again.
Rate limits and timeouts are another. Model providers return rate-limit errors under load, and responses can be slow. The retry and backoff logic, and whatever happens when retries run out, usually only runs in production on a busy day.
Fallbacks deserve a close look too. Many apps switch to a smaller model, a cached answer or a polite apology when the main model fails. If that fallback path has never run, nobody actually knows what users see when it kicks in.
And then there are guardrails. Content filters, checks for leaked personal data, rules that block certain actions. These are some of the most important branches in the whole feature, and since normal conversations rarely trigger them, they’re also some of the least tested.
What white box testing adds here
White box testing looks inside the code and measures which lines and branches actually execute. For an LLM-powered feature, that turns a vague worry into a concrete list.
Run your existing tests with branch coverage switched on and look specifically at the code around the model call. In many AI features, the result is pretty striking. The happy path is covered over and over, while the parsing error handler, the retry exhaustion branch and the guardrail rejection path show as never executed. That’s not a judgment on the code’s quality. It’s simply a list of behavior nobody has checked yet.
Making the model’s behavior repeatable
There’s an obvious problem with testing code around a model: the model doesn’t give the same answer twice. Outputs can vary even with randomness turned down, and calling a real model on every test run is slow and costs money. Tests that depend on live model calls tend to be flaky, and flaky tests make coverage numbers meaningless.
The fix is recording real model responses once and replaying them during tests. The surrounding code gets exactly the same answer every run, so any change in behavior points at the code, not the model. Recording at the network level also means the application doesn’t need special test hooks, which keeps the code under test identical to what runs in production.
Recorded responses are also a great starting point for the tricky cases. Take a real, valid response and edit it into a broken one: cut it off halfway, remove a required field, wrap it in extra text. Turn a successful call into a rate-limit error or a timeout. Now the parsing handler, the retry logic and the fallback path all get a chance to run, on demand, with realistic data around them.
Reading the coverage report
Once recorded responses drive the tests, the coverage report becomes a useful map of the feature’s safety.
Prompt assembly branches should all show as covered, including the ones for empty retrieval results and conversations that need trimming. Every error path around the model call should have run at least once. Guardrail branches deserve special attention, since an untested filter is one you’re trusting blindly. And any tool or action the model can trigger, like sending an email or updating a record, should have its permission checks covered.
Coverage proves code ran, not that it behaved correctly. So for each branch, it’s worth checking what the user actually saw. A fallback that runs but shows a confusing error isn’t much of a fallback.
Keeping it current as prompts change
AI features change constantly. Prompts get tweaked, models get upgraded, new guardrails get added. Each change can shift which branches run and how.
A good habit is re-recording a set of real model interactions whenever the prompt or model changes, then replaying them with coverage on. If a branch that used to run stops running, or a new one appears that nothing exercises, the change shows up in review. Prompt changes go from untested edits to something the pipeline actually checks.
Conclusion
The model gets most of the attention in AI features, but the code around it decides whether users ever see a broken answer, an endless spinner or a leaked detail. White box testing shines a light on that surrounding code, and recorded model responses make it possible to test it consistently, failures included.
Teams shipping AI features don’t need to understand the model’s internals to make them safer. They need to know that every branch they wrote for the moment things go wrong has actually run at least once, and recorded responses make that a routine check instead of a hope.
Sign in to leave a comment.