Regression gate
A regression gate is a CI check that fails a build when an evaluation suite’s pass rate drops below a defined threshold, preventing a prompt or model change from shipping a quality regression.
The gate is what converts evals from a reporting exercise into a control. Teams that run evals manually stop running them within a month. Teams whose merge is blocked by a failing suite keep the suite healthy, because it is now in the path of everything they do.
Related terms
- Eval harness
An eval harness is the test infrastructure for a non-deterministic AI system: a dataset of representative inputs, a method for scoring each output, and a threshold that determines whether a run passes or fails.
- Golden dataset
A golden dataset is a curated set of inputs paired with known-correct outputs, used as the fixed reference against which an AI system’s quality is measured over time.
Tell us what you are building
Send us the problem in a paragraph. You will get a straight answer on whether we can help, and what we would do first.
Book a call