Topics / Engineering practice
A crash your suite missed is a new verification obligation.
First preserve the failure. Then find the missing condition, fix the instance and decide how much of the wider failure class needs checking.
Keep a failing witness before changing the code.
Record the input, environment, revision and observed crash. Minimize the input when possible, while confirming the smaller case still demonstrates the same failure. If you cannot reproduce it, retain the attempted conditions and uncertainty.
Illustrative example
An empty batch reaches a non-empty assumption.
Production sends a batch containing no valid records. A worker indexes the first record and panics. Existing tests all construct at least one valid record. The repair needs a decision about an empty batch: reject it, accept it as a no-op, or report partial failure.
Fix the behavior, not just the symptom.
Returning early removes the panic but may silently skip an obligation. Tie the repair to the intended outcome, then execute the original case and neighboring cases: all-invalid input, a mixed batch and an interrupted retry.
- Retain the original failing revision and reproducer.
- Record the chosen empty-input behavior and authority.
- Run the regression on the fixing revision.
- Review sibling entry points with the same assumption.
- Keep class-level gaps open if they were not checked.
Turn the incident into reusable context.
Proof distinguishes an unresolved known issue from the evidence-bearing defect record retained after verified closure. A passing reproducer supports an instance-level conclusion; broader closure needs its own evidence.
That record lets a later agent understand why the guard exists and which behavior it protects. Do not leave the lesson only in a closed incident chat.
Try it on your software
Bring one escaped crash and its smallest reproducer.
We can trace it to the missing obligation and build a closure record the next change can reuse.