Flip test alignment

The check is flip_test_alignment. It is VERIFY-stage. Default severity is warning. It always runs. It only looks at components listed in project.verification.fixture_evidence.components. Then the hop asks whether requirements that own FLIP fixtures also have source-native Verifies: test evidence. It does not ask whether the files exist. That hop stays on flip fixtures exist. It does not ask whether those files still match the FRETish. That hop stays on fixture staleness clean. The instrument stays on MC/DC coverage for Go.

Empty fixture_evidence.components is a pass: no components declare fixture-backed evidence -- FLIP test alignment not applicable. No tests/ directory is also a pass: no tests/ directory -- FLIP test alignment not applicable. No FLIP rows on the listed components is a pass: no FLIP fixtures found for declared fixture-evidence components -- test alignment not applicable. Those passes are silence, not a coverage proof. A tests/ path the hop cannot read, a requirements load error, or an alignment evaluation that cannot run, is fail: the hop cannot decide whether the rows have tests. When opted-in FLIP rows have no source-native test, the hop warns: N requirements with FLIP fixtures, M without test coverage. The detail names the uncovered scenarios and the signal combination from the fixture rows: SYS-REQ-… has N FLIP scenarios but no test. Write a test covering: …. A requirement that already has a non-fixture VerifiedBy is not in that list. Undeclared components may keep FLIP JSON as helper artifacts. The hop does not treat those files as controlled evidence.

The expensive miss is a review that treats generated truth-table rows as the pack, then a green existence hop, then no test that actually drives the Go. Existence still closes, because the files are still there. Freshness can still close, because the fingerprint still matches. This hop is the gate that says a listed component’s FLIP rows have a matching test, or names the requirements whose scenarios never left the JSON.

Typical uncovered rows are a named scenario plus the signal combination in that fixture. Adding a test without Verifies: is not alignment. Deleting the JSON to silence the checker is not a test. Removing the component from the list is the honest move only when the rows were never a review artifact.

# proof.yaml
project:
  verification:
    fixture_evidence:
      components: [firewall]

# tests/firewall/tc-001.json exists
# no Go test with Verifies: SYS-REQ-firewall-quota
# proof audit --check flip_test_alignment
# 1 requirements with FLIP fixtures, 1 without test coverage
# SYS-REQ-firewall-quota has 1 FLIP scenarios but no test.
# Write a test covering: quota-full (over_limit=true)
# warning: the rows are not a test

The fix is to write a source-native test for the named scenarios, with Verifies: on that requirement, or to drop the component from the list if the JSON was never controlled evidence. Inspect first: proof mcdc show <requirement-id>. Use the warning detail as the initial matrix. Then re-run VERIFY. Do not copy a sample JSON into the suite and call it coverage. A quiet hop after you ignore the warning is still not alignment. The default warning does not block advancement.

proof audit --check flip_test_alignment --verbose
proof mcdc show SYS-REQ-firewall-quota
proof help flip_test_alignment

The teaching graph is still one guarantee whose scenario files still exist while no test covers those rows. Close it by writing the named tests, or by removing the component from fixture_evidence.components if the JSON was never a review artifact. Do not delete the JSON to silence the checker. With no opted-in component the empty pass still prints, and the alignment question did not go anywhere. It just was never in scope.

# source-native test for the opted-in component
# // Verifies: SYS-REQ-firewall-quota
# TestFirewallQuotaOverLimit covers quota-full (over_limit=true)

# proof audit --check flip_test_alignment
# 1 requirements with FLIP fixtures, all have test coverage
#
# VERIFY is allowed to move on. a pass now sits on a named test

The existence hop stays on flip fixtures exist. The freshness hop stays on fixture staleness clean. The Z3-example hop stays on property fixtures exist. The instrument hop stays on MC/DC coverage for Go. The rewrite hop stays on characterization testing and mirrors. Jama still authors. Proof vs Jama.

Warning when opted-in FLIP rows have no source-native Verifies: test. Fail when requirements cannot be loaded, when tests/ cannot be read, or when alignment cannot be evaluated, so the hop cannot decide. Empty fixture scope is a pass, not a fail: the hop does not invent tests. No tests/ directory is a pass, not a fail. No FLIP rows on the listed components is a pass, not a fail. The hop does not write the test. It does not add a shall. It does not prove the Go. It does not run Kind2. It does not measure independence. A quiet hop is not a proof that the authored guarantee is the one you meant, only that every opted-in FLIP row has a matching test, or that none were in scope. It does not say the files exist. That existence hop is flip fixtures exist. It does not say the files still match the FRETish. That freshness hop is fixture staleness clean. The default warning does not block advancement. We have not scored this floor against a frozen Jama pack or a VectorCAST corpus. The loss is named, not scored.

The existence hop stays on flip fixtures exist. The freshness hop stays on fixture staleness clean. The rewrite hop stays on characterization testing and mirrors. The engagement stays on software correctness audit. Jama still authors.

What is flip test alignment? Same question. Same URL.

Is this flip fixtures exist? No. That hop is whether opted-in tc-*.json files exist. This hop is whether those rows have a source-native test. See flip fixtures exist.

Is this fixture staleness clean? No. That hop is whether existing files still match the formalization. This hop is whether a test covers the rows. See fixture staleness clean.

Is this property fixtures exist? No. That hop is whether opted-in Z3 examples exist. This hop is FLIP-to-test alignment. See property fixtures exist.

Is this MC/DC coverage? No. That hop is whether each condition independently affects the decision. This hop is whether generated FLIP obligations have a test. See MC/DC coverage for Go.

Is this characterization testing? No. That page is rewrite-with-a-ledger. This hop is FLIP-to-test alignment for a listed component. See characterization testing and mirrors.

Does a quiet hop prove the Go matches the shall? No. The hop observes traces. It does not prove the Go.

Does empty fixture_evidence fail this hop? No. That is a pass. A pass with nothing in scope is not alignment.

Does no tests/ directory fail this hop? No. That is a pass: FLIP test alignment is not applicable.

Does a missing Verifies: test fail the merge? No. The check keeps warning severity unless you raise it in proof.yaml.

Is Proof an LDRA alternative for the instrument? No. LDRA still owns the avionics toolchain. Proof vs LDRA.

Is Proof a Jama alternative for the shall? No. Jama still authors. Proof vs Jama.