The same passing test can support two different claims
Illustrative example. A reservation service must retain retry keys for 24 hours. A test repeats a request after 23 hours and confirms that no second reservation is created. The values below describe a hypothetical run, not a published execution.
| Basis | Revision A | Revision B |
|---|---|---|
| Approved requirement | SW-REQ-241 revision 2: retain keys for 24 hours. | Unchanged. |
| Service implementation | Commit A. | Commit A. |
| Deployed configuration | Key retention: 24 hours. | Key retention: 12 hours. |
| Recorded check | Retry at 23 hours; one reservation observed under A configuration. | No run under B configuration. |
| Conclusion | Supports this retry scenario under the recorded conditions. | Earlier pass does not establish behavior with 12-hour retention. |
No source file changed. The configuration did. Reusing the earlier green result would hide exactly the behavior the requirement is meant to protect. Rerun the scenario with the new configuration; if the key has expired and a duplicate appears, the new result is a violation, not merely stale evidence.
Check the basis that can change the conclusion
| Changed basis | What needs attention |
|---|---|
| Requirement or model | Check whether the old assertions and generated fixtures still represent the approved behavior. |
| Implementation, dependency or relevant base revision | Identify affected paths and rerun the checks whose behavior may have changed. |
| Configuration, data or environment | Compare values that influence the result: retention, feature flags, schema, runtime and dependency versions. |
| Acceptance policy or exception | Reevaluate readiness and authorization. A test can remain valid while no longer satisfying the acceptance policy. |
| New failure or threat observation | Review the assumptions and coverage, even if every recorded input hash is unchanged. |
An unrelated documentation correction need not invalidate a behavioral test. Conversely, a relevant new incident may undermine a broad conclusion without changing any file. Record the reason for renewal or reuse so another reviewer can examine it.
Keep the old run and renew the affected evidence
- Name the claim. Identify the requirement revision and the scope the result was used to support.
- Compare relevant inputs. Include the test itself, fixture data, configuration, environment and applicable policy.
- Separate states. Mark affected evidence stale or insufficient; keep the previous run as historical evidence. Do not label it a new failure until an observation supports that conclusion.
- Run the missing checks. Regenerate derived fixtures where needed, inspect changed expectations and execute against the candidate revision.
- Reevaluate acceptance. Bind the renewed decision to that evidence. An older job completing later must not replace the current result.
Claim: SW-REQ-241 revision 2, retry at 23 hours.
Previous run: code A + test T4 + config C24 + environment E2.
Changed basis: config C12 reduces retention below the requirement.
Historical result: pass, retained for the previous basis.
Current evidence: missing for code A + test T4 + config C12 + E2.
Next action: execute the scenario with C12; inspect reservation count.
Acceptance: pending renewed evidence and disposition.Which freshness checks Proof currently supplies
Proof has specific freshness checks, with specific scopes. Generated-fixture checking compares the inputs used to create fixtures; interface checks compare recorded artifact fingerprints. Neither is a universal detector for every configuration change, new threat or policy decision.
The fixture freshness check is disabled by default and applies to components explicitly declared for controlled fixture evidence. Read the installed help before enabling it:
proof help fixture_staleness_clean
proof help interface_staleness_cleanRegenerating a fixture does not execute its behavioral check. A workflow warning also does not, by itself, prohibit merging: required CI checks, revision binding and bypass permissions determine enforcement.
Use the broader basis above as a review procedure where automated checks do not cover it. The verification report should identify those manual comparisons and remaining gaps.