Topic · Property fixtures exist

Property fixtures exist

Gist

A Z3 example you opted into and never generated is missing evidence. Proof runs proof audit --check property_fixtures_exist. A proved property is not that hop. Jama still authors.

proof audit --check property_fixtures_exist

Keep the unit tests if they still catch the samples. Keep Jama if it already holds the shall. Neither one asks whether tests/<component>/z3-*.json exists for a component you listed as fixture-backed.

01 · The silent empty pass

Verify can print clean because examples were never in scope.

You can ship a merge algebra in YAML, prove it in Z3, and still have no generated examples. This hop stays quiet until you opt a component in.

The check is property_fixtures_exist. It is VERIFY-stage. Default severity is warning. It is disabled by default. Enable it with project.checks.property_fixtures_exist.enabled: true, and list the component in project.verification.fixture_evidence.components. Then the hop asks whether Z3-derived example files exist. It does not prove the specification. The proof gates stay on Z3 properties verified, data constraint Z3 coverage, and behavioral implications verified.

Empty fixture_evidence.components is a pass: no components declare fixture-backed evidence -- property fixtures not applicable. Opted-in components with no Z3-backed properties: or data_constraint is also a pass. Those passes are silence, not a fixture set. A variable directory or .vars.yaml parse error is fail: the hop cannot decide whether fixtures apply. Missing tests/, or no z3-*.json under an opted-in component, is warning.

The expensive miss is a review that treats generated examples as part of the pack, then CI that never wrote them. The audit then reports that every property was fine, because the proof hop closed, or because this hop never ran. This hop is the gate that says the examples exist, or names the variables that still have none.

This is not whether the authored merge proved. That hop is Z3 properties verified. A proved subject can still have zero example files. The help file teaches that first.

# proof.yaml
project:
  checks:
    property_fixtures_exist:
      enabled: true
  verification:
    fixture_evidence:
      components: [firewall]

# tests/firewall/ is empty
# proof audit --check property_fixtures_exist
# 1 variables with Z3-backed properties or data constraints
# across declared fixture-evidence components but no Z3 fixture
# files (z3-*.json) in tests/
# warning: the examples were never generated

The fix is to generate the examples, or to drop the component from the list if they were never a review artifact. Run proof proptest specs/system <component> --source z3 --output tests/<component>/. Then re-run VERIFY. Do not copy a sample JSON by hand to silence the checker. A quiet hop after you ignore the warning is still not a fixture set. The default warning does not block advancement.

proof audit --check property_fixtures_exist --verbose
proof proptest specs/system firewall --source z3 --output tests/firewall/
proof help property_fixtures_exist

02 · The exhibit

Same merge algebra. Silent empty pass, or this hop.

One last green VERIFY on firewall. Z3 proved the merge. Nobody wrote z3-*.json. Click the tabs.

The stamp

  • Ask did the authored merge prove
  • Stamp z3_properties_verified last pass. tests/firewall empty
  • Why the proof hop closed. nobody asked for examples
Proof hop green

This hop

Nobody asked whether the opted-in examples exist. A proved merge is not a z3-*.json file. The finding kind is this hop.

Need unread

The stamp

Keep the live proof. Keep the Jama cell. That is not this hop.

Keep the record

Proof

  • Ask do opted-in Z3 example files exist
  • Out property_fixtures_exist, 1 Z3-backed vars, 0 z3-*.json
Examples missing

Same merge algebra. Silent empty pass, or this hop. Click the tabs.

Surface What they do What Proof does What we lose
Green tests The last samples that still unioned. Whether opted-in z3-*.json files exist under tests/. We do not rerun the suite here. A green stamp is not this hop.
Z3 properties verified Whether the authored merge proved. Whether examples exist after you opted the component in. Not the proof. See Z3 properties verified.
Property based testing The technique, and fixtures kept true in CI. This named hop: optional Z3 examples for listed components. Not the technique page. See property based testing.
Hypothesis / QuickCheck A generator and a shrinker on sampled inputs. Concrete Z3 examples written as z3-*.json. We do not replace Hypothesis. We have not run a frozen Hypothesis corpus.
Jama cell A shall, and a parent if you type it. A warning the audit can name next to the missing examples. Not Jama's V&V. Jama still authors. We have not run a frozen Jama pack.

The teaching graph is still one output whose merge proved while tests/firewall/ stayed empty. Close it by generating the examples, or by removing the component from fixture_evidence.components if they were never a review artifact. Do not add another sample only to silence the checker. With no opted-in component the empty pass still prints, and the examples did not go anywhere. They just were never in scope.

# examples generated for the opted-in component
# proof proptest specs/system firewall --source z3 --output tests/firewall/
# wrote tests/firewall/z3-allowed_hosts-union.json

# proof audit --check property_fixtures_exist
# 1 property fixtures across 1 components (1 Z3-backed vars)
#
# VERIFY is allowed to move on. a pass now sits on files

The proof hop stays on Z3 properties verified. The skip-list hop stays on data constraint Z3 coverage. The implication hop stays on behavioral implications verified. The technique page stays on property based testing. Jama still authors. Proof vs Jama.

03 · The honest loss

Proof names the missing examples. It does not write them, and it does not prove the Go.

A quiet proof audit --check property_fixtures_exist can still mean the hop was never enabled. Jama still authors.

Warning when opted-in Z3-backed variables have no z3-*.json, and when tests/ is missing. Fail when a variable file cannot be read, so the hop cannot decide whether fixtures apply. Empty fixture scope is a pass, not a fail: the hop does not invent examples. Disabled by default: if you never enable it, CI never asks. The hop does not write the JSON. It does not add a shall. It does not prove the Go. It does not run Kind2. A quiet hop is not a proof that the authored merge is the one you meant, only that every opted-in component has example files, or that none were in scope. It does not count skipped data_constraint rows. That count is the other hop. The default warning does not block advancement. We have not scored this floor against a frozen Jama pack or a Hypothesis corpus. The loss is named, not scored.

The proof hop stays on Z3 properties verified. The skip-list hop stays on data constraint Z3 coverage. The implication hop stays on behavioral implications verified. The technique page stays on property based testing. The engagement stays on software correctness audit. Jama still authors.

04 · Nearby questions

What people type next.

What is property fixtures exist? Same question. Same URL.

Is this Z3 properties verified? No. That hop is whether the authored merge proved. This hop is whether opted-in example files exist. See Z3 properties verified.

Is this property based testing? No. That page is the technique. This hop is the optional Z3-example gate. See property based testing.

Is this data constraint Z3 coverage? No. That hop is whether an authored case was even lowered. This hop is whether examples exist after you opted in. See data constraint Z3 coverage.

Is this behavioral implications verified? No. That hop is whether a lowered implication holds. This hop is files under tests/. See behavioral implications verified.

Does a quiet hop prove the merge is the one you meant? No. The hop observes example files. It does not score the sentence.

Does a quiet hop prove the code matches the shall? No. The hop observes files. It does not prove the Go.

Does empty fixture_evidence fail this hop? No. That is a pass. A pass with nothing in scope is not a fixture set.

Does missing z3-*.json fail the merge? No. The check keeps warning severity unless you raise it in proof.yaml.

Is Proof a Jama alternative for the shall? No. Jama still authors. Proof vs Jama.