Your agent worked yesterday. Then someone tightened a prompt, the model provider shipped an update, or a tool changed its output format, and now it quietly fails one in five of the cases it used to handle. Nobody saw it happen. That is the core problem with shipping agents: they degrade silently, and you usually find out from a user, not a dashboard.

Regression testing is the fix. In plain terms, it means re-running a fixed set of known-good cases, a golden set, after every change, and refusing to ship if the pass rate drops below a bar you set. The discipline is old; software teams have leaned on it for decades. What is new is adapting it to agents, where the same input can return different words each time and where the model underneath you can shift without warning.

This guide walks through what regression testing means for agents, why they regress, how to build a golden set, how to handle non-determinism, how to wire it into CI with a pass bar, and how to triage failures. It builds on AI agent reliability testing explained and pairs with how to test an agent before deploy.

What regression testing means for AI agents

Regression testing for an agent means keeping a fixed golden set of cases and re-running it after every change, then blocking the change if the pass rate falls. The golden set encodes behavior you already trust. A new prompt, model, or tool ships only when the agent still clears your pass bar, so quality cannot quietly slip backward between releases without a test going red first.

The idea maps the classic software practice onto a probabilistic system. Microsoft frames evaluating LLM applications as a continuous loop rather than a one-time check, run during development and again before each release, precisely because behavior shifts over time (Microsoft Learn, "Evaluating generative AI applications", 2024). Regression testing is the part of that loop that protects what already works. The mental shift that trips engineers up: you are not asserting one correct output, you are defending an aggregate pass rate.

How it differs from a one-off eval

A one-off evaluation tells you how good the agent is today. Regression testing tells you whether your latest change made it worse. The first is a snapshot; the second is a comparison against a known baseline. You need both, but only the comparison stops a quiet drop from reaching users. For the broader scoring picture, see the AI agent evaluation framework, step by step.

Why agents regress

Agents regress because three things underneath them keep moving: the model, the prompt, and the tools. Any one can change behavior with zero code errors and no exception in your logs. Anthropic's guidance is blunt about this: agent systems trade speed and cost for autonomy, so you should test extensively and add guardrails, because small changes ripple (Anthropic, "Building Effective Agents", 2024).

Model swaps and silent provider updates

Swap to a newer or cheaper model and the whole behavior profile shifts. It may follow instructions differently, format output another way, or refuse cases it used to handle. Worse, hosted models can change under a stable name, so an agent you never touched can regress overnight. That is why a scheduled golden-set run matters as much as a per-change one.

Prompt edits and tool or version changes

A prompt is code. Reword one instruction to fix one case and you can break three others you forgot you cared about. In our experience, the most common regression source is a well-meaning prompt tweak that fixes the bug in front of you and silently dents an edge case nobody re-checked. Tools regress too: an API changes a field name, a library bumps a version, retrieval returns a new ranking, and the agent reasons over different inputs than before. A/B testing strategies for agents help when you want to compare two versions live rather than just guard a baseline.

Build a golden test set

A golden set is a curated collection of real cases, each storing an input, the expected behavior or graded criteria, and a label. Start small and grow it from production failures rather than trying to write hundreds up front. Anthropic recommends building evaluations from real usage and adding cases as you find problems, so each fix becomes permanent protection (Anthropic, "Building Effective Agents", 2024).

What to put in it

Cover three buckets. First, common happy paths, the work the agent does all day. Second, the tricky edge cases that broke before, because those are your highest-value tests. Third, adversarial and unsafe inputs the agent must refuse or escalate. Every time a bug reaches production, add it as a case. Our own internal suite started near a dozen cases and grew past 80 as we converted each real failure into a permanent test, which is the story behind how we test AI agents with 80 tests.

How to label expected behavior

Avoid storing one exact correct string. Store the properties that must hold: a required fact, a forbidden claim, a structure the output must follow, or a rubric a grader applies. This keeps cases valid even as wording shifts. For the scoring side in depth, see AI agent quality scoring methods.

Handle non-determinism

The same prompt can produce different output every run, so exact-match assertions are the wrong tool. Google's guidance on testing generative systems stresses checking for desired qualities and behaviors rather than a single fixed answer, because outputs vary by design (Google for Developers, machine learning guides). You test that the behavior is correct, not that the text is identical.

Tolerances and multiple runs

Run each case several times instead of once, then judge the aggregate. A single run can pass or fail by luck; five runs give you a rate you can trust. Set tolerances rather than equality: the answer must contain the right figure, stay within a length band, or call the expected tool, not match a reference word for word. Flaky cases that pass three times in five tell you something real about consistency.

Rubric and LLM-as-judge scoring

For open-ended output, score against a rubric. A grader, sometimes another model acting as judge, rates each output on criteria like correctness, completeness, and safety. The catch most teams miss: your judge is itself non-deterministic and can regress, so spot-check it against human labels on a sample, or your safety net has a hole in it. Keep deterministic checks, exact facts and refusals, as hard assertions, and reserve rubric scoring for the fuzzy parts.

Automate it in CI with a pass bar

Manual testing does not survive a busy week, so the golden set has to run automatically. Wire it into CI so every change to the prompt, model, tools, or retrieval data triggers the suite before merge, and gate the release on a pass bar. Microsoft describes exactly this pattern: evaluate continuously and before each release so regressions are caught pre-deployment (Microsoft Learn, 2024).

Set the pass bar against your baseline

There is no universal passing number. Anchor the bar to your own last green run: block any change that drops the golden-set pass rate below the prior baseline, and hard-fail on any regression in safety or refusal cases. Treat safety cases as must-pass. Treat quality cases as a threshold you raise as the suite matures. How to test an agent before going live walks the final pre-launch gate.

A minimal CI shape

The structure stays simple even when the agent is complex. You load the golden set, run each case the chosen number of times, score, aggregate to a pass rate, and exit non-zero if it falls below the bar so the pipeline blocks the merge.

load golden_set            -> N cases (happy, edge, safety)
for case in golden_set:
    run case x5            -> collect outputs
    score(outputs)         -> rubric + hard checks
pass_rate = passed / total
if pass_rate < baseline:   exit 1   # block the release
if any safety_case failed: exit 1   # hard fail, no exceptions

Triage and fix regressions

A red run is information, not a verdict, so triage before you revert. Anthropic's advice to maintain visibility into agent steps applies directly here: trace the failing case to the stage that changed, the prompt, the model call, or the tool, so you fix the cause rather than patch the symptom (Anthropic, "Building Effective Agents", 2024). Read the actual failing outputs first.

Decide: real regression or stale test?

Sometimes the agent got genuinely worse. Sometimes the expected behavior legitimately changed and the test is now stale. Both look like a red run. Read the output, decide which it is, and either fix the agent or update the case with a clear reason in the commit. Never loosen a test just to make CI green; that is how a real regression slips through.

Close the loop permanently

Whatever broke, add or sharpen a case so the same regression cannot return silently. This is the compounding habit: each fix makes the suite stronger, and over months the golden set becomes the single best record of what your agent must do. If you are starting from zero, how to set up your first AI agent covers the groundwork before you layer testing on top.

Frequently asked questions

What is regression testing for an AI agent?

Regression testing means re-running a fixed golden set of cases after every change to your agent and checking that the pass rate did not drop. The golden set captures known-good behavior. A change ships only if the agent still clears the pass bar, so prompt edits, model swaps, and tool updates cannot quietly make the agent worse.

How do I regression test a non-deterministic agent?

Stop asserting exact strings. Run each case several times, score outputs against a rubric or check key facts and structure, and set tolerances instead of equality. Aggregate to a pass rate over the whole golden set and gate on that. The same input can produce different wording, so you test for correct behavior, not identical text.

How often should I run agent regression tests?

Run the full golden set in CI on every change to the prompt, model, tools, or retrieval data, before it reaches users. Run a smaller smoke subset on each commit for fast feedback. Because model providers update hosted models over time, also schedule a periodic run so a silent upstream change gets caught even when your own code did not move.

What should go in an agent golden set?

Real cases you care about: common happy paths, the tricky edge cases that broke before, and adversarial or unsafe inputs the agent must refuse. Each case stores the input, the expected behavior or graded criteria, and a label. Add every new bug as a case so the same regression cannot return. Start small and grow it from production failures.

What pass bar should gate an agent release?

Pick a bar from your own baseline, not a universal number. A common pattern is to block any release that drops the golden-set pass rate below the last green run, and to hard-fail on any regression in safety or refusal cases. Treat safety cases as must-pass and quality cases as a threshold you tune as the set grows.

The Gravity way

On a platform like Gravity you do not run regression suites yourself. Gravity maintains its expert-built agents and re-tests them when prompts, models, or tools change, so the agent you use stays at the quality you expect. You describe the outcome you need and the right agent handles it in about 60 seconds. You pay only when it runs, at $1 for 1,000 credits, with the testing burden kept on the platform side.

Sources