Run the same check with a shorter retention window
The executed teaching fixture keeps retry keys for 24 hours at revision A. At B, only the configuration changes to 12 hours. The implementation, requirement and five expected results remain unchanged. At C, retention returns to 24 hours.
| Basis or observation | A | B | C |
|---|---|---|---|
| Retention configuration | 24 hours | 12 hours | 24 hours |
| Retry at 23 hours | 1 reservation | 2 reservations | 1 reservation |
| Direct test and Proof test-check exit | 0 / 0 | 1 / 1 | 0 / 0 |
| Proof package readiness | not_assessed | blocked | not_assessed |
Before B runs, A’s pass is historical evidence for the 24-hour setup. After B runs, its duplicate reservation is an observed violation. Keep both facts. Restoring retention repairs these cases; it does not resolve every acceptance gap.
The fixture author explicitly identifies the configuration dependency. A small runner compares the hashes of its declared inputs; this is not automatic dependency discovery by Proof. The test uses an injected logical clock, not a 23-hour wait, and the requirement remains a draft teaching record. Inspect exact revisions, commands and hashes.
Check the basis that can change the conclusion
| Changed basis | What needs attention |
|---|---|
| Requirement or model | Check whether the old assertions and generated fixtures still represent the approved behavior. |
| Implementation, dependency or relevant base revision | Identify affected paths and rerun the checks whose behavior may have changed. |
| Configuration, data or environment | Compare values that influence the result: retention, feature flags, schema, runtime and dependency versions. |
| Acceptance policy or exception | Reevaluate readiness and authorization. A test can remain valid while no longer satisfying the acceptance policy. |
| New failure or threat observation | Review the assumptions and coverage, even if every recorded input hash is unchanged. |
An unrelated documentation correction need not invalidate a behavioral test. Conversely, a relevant new incident may undermine a broad conclusion without changing any file. Record the reason for renewal or reuse so another reviewer can examine it.
Keep the old run and renew the affected evidence
- Name the claim. Identify the requirement revision and the scope the result was used to support.
- Compare relevant inputs. Include the test itself, fixture data, configuration, environment and applicable policy.
- Separate states. Mark affected evidence stale or insufficient; keep the previous run as historical evidence. Do not label it a new failure until an observation supports that conclusion.
- Run the missing checks. Regenerate derived fixtures where needed, inspect changed expectations and execute against the candidate revision.
- Reevaluate acceptance. Bind the renewed decision to that evidence. An older job completing later must not replace the current result.
Claim: SW-REQ-241, retry at 23 hours (worksheet).
Previous run: code A + test T4 + config C24 + environment E2.
Changed basis: config C12 reduces retention below the requirement.
Historical result: pass, retained for the previous basis.
Current evidence: missing for code A + test T4 + config C12 + E2.
Next action: execute the scenario with C12; inspect reservation count.
Acceptance: pending renewed evidence and disposition.Which freshness checks Proof currently supplies
Proof has specific freshness checks, with specific scopes. Generated-fixture checking compares the inputs used to create fixtures; interface checks compare recorded artifact fingerprints. Neither is a universal detector for every configuration change, new threat or policy decision.
The fixture freshness check is disabled by default and applies to components explicitly declared for controlled fixture evidence. Read the installed help before enabling it:
proof help fixture_staleness_clean
proof help interface_staleness_cleanRegenerating a fixture does not execute its behavioral check. A workflow warning also does not, by itself, prohibit merging: required CI checks, revision binding and bypass permissions determine enforcement.
Use the broader basis above as a review procedure where automated checks do not cover it. The verification report should identify those manual comparisons and remaining gaps.