Topics / Engineering practice
Your suite can pass while a customer promise is untested.
A test suite is a collection of observations. Find the requirements it does not exercise before interpreting a green run as confidence in the whole change.
Start from the promise, not the test count.
List the behaviors a change must preserve. For each, inspect whether a relevant test has an assertion that would fail if that promise were broken. A requirement ID in a comment helps trace the test; it does not establish the strength of its assertions.
Illustrative example
The import returned 200 and lost a row.
The integration test asserts success and checks that the response parses. The customer promise is that every accepted input row is stored exactly once. Inspect row identity and count after a retry. That missing assertion matters more than another execution of the status-code check.
Look for three kinds of gaps.
- Missing behavior: no test represents the promised outcome.
- Weak observation: a test runs the path but does not assert the consequential effect.
- Missing condition: the check covers normal input but not a relevant boundary, retry or failure.
Coverage can help locate unexecuted code. It cannot by itself tell you whether an executed path did the right thing. Use both structural and behavioral evidence.
Keep the mapping honest.
Proof links requirements and test evidence so missing support can be inspected. The quality of the expected outcomes and the correctness of the links still need review. Recorded coverage is useful only for its identified revision and execution.
After finding a gap, retain the new requirement or clarified expectation and the regression check. The next agent should inherit the reason the check exists.
Try it on your software
Choose a green PR that still feels under-tested.
We will compare its customer promises with the observations the suite actually makes.