◄ Wiki / Build
A/B test
Running the same suite twice — feature off, then on — to measure exactly what the feature changed.
An A/B test isolates a single variable by comparing two otherwise-identical runs, so any difference in results is attributable to that one change. In software validation this means executing a full test suite with a feature disabled and again with it enabled — for example a 2556-case A/B — and diffing the outcomes. It separates real effects from noise or pre-existing failures.
See also
- A/B validationChecking your implementation by running the same work through a trusted reference and comparing outputs.
- assertionA runtime check that says 'this must be true here,' and complains loudly if it isn't.
- bit-exactProducing output identical to a reference down to every last bit.