A customer reported a crash our whole test suite missed. How do I prevent that entire class?

The recurrence question lives on why the same bugs come back. That page is a door that re-opens after a fix. This page is the first report: the suite never had the door, because nobody wrote the promise the crash violated.

Kind2 realizability answers a different question from a unit test. A test asks whether this input survived. proof realize asks whether there exists any implementation that can keep the approved shalls. If the shalls contradict, or if a guarantee is false for some input the component cannot refuse, the solver returns a counterexample. That counterexample is a class, not a ticket.

proof realize specs/system ./... --diagnose
proof realize specs/system ./... --format json

UNREALIZABLE is not a modeling insult. It is the honest residue: the shall cannot be kept, or it is a reachability obligation the component cannot self-enforce. The diagnose flag narrows the conflict to one requirement. See proof help tracked-violations when the term only mentions inputs.

If the job is “tell me when production panics,” keep Sentry. If the job is “are my tests sensitive to this edit,” keep PITest. If the job is “this class of input must not crash, including spellings nobody typed,” those tools are the wrong instrument. That is the check we run.

The hazard sweep matched keys[i][0], fixed seven sites, and signed sibling_sweep: result: clean. The eighth site was written keys[depth:][0][0]. Same class. Different spelling. The suite did not construct it. Coverage still counted the function. The record is on the jsonparser miss, not as a slogan here.

A different story on the same library: OSS-Fuzz once found a panic in Delete by mutating bytes. Encoded as a solver constraint against the formalized spec, the same hole fell out as a counterexample in milliseconds. Random search found an instance. Once specified, the class is a query. That does not mean every class is already specified. The two published misses are the proof of that sentence.

proof realize specs/system ./... --diagnose

Closing the class takes more than one passing reproducer. The miss protocol writes the missing shall, pins a reproducer that fails without the fix, and sweeps siblings. One green test is the instance. The shall in CI is the class. That protocol is on life after the audit.

We have not run Proof and PITest, or Proof and Sentry, on the same frozen corpus. A green proof realize means Kind2 did not find a counterexample on the approved shalls. It does not mean every crash class a customer could hit is in the graph. The jsonparser misses are published because that sentence has to stay honest.

Mutation testing stays on the recurrence page. A Z3 lemma on one function stays on Z3 / Kind2. Jama still wins at programme authoring. Proof will not replace a fuzzer, a crash inbox, or an ALM. The commercial shape of the install is on the audit page.

How do I find the class of bugs my tests never check for? Write the shall. Run proof realize. That is this URL, not a second page.

Why do the same bugs keep coming back after we fix them? That is a door with no pin. Why bugs come back.

We have high coverage and still ship escaped defects. Why? Coverage counts lines that ran. The recurrence page owns that head. Do not mint a twin.

How do I prove a specific function with Z3 or Kind2? A lemma on the function is a different H1. Z3 / Kind2.