The stamp
- Ask did the two sampled hosts still union
- Stamp allowed_hosts last pass. no Z3 on the runner
- Why the samples are covered. nobody asked every input
Topic · Z3 properties verified
Gist
A properties: block that nobody proved is a comment. Proof runs proof audit --check z3_properties_verified. A green suite on sampled inputs is not that hop. Jama still authors.
proof audit --check z3_properties_verified
Keep the unit tests if they still catch the samples. Keep Jama if it already holds the shall. Neither one asks Z3 whether the authored merge is true for every input.
01 · The silent empty pass
You can ship a merge algebra in YAML and never run Z3. Lint still prints clean. The hop still prints pass when no variable carries a Z3-backed property or constraint.
The check is z3_properties_verified. It is VERIFY-stage. Default severity is warning. The hop asks whether Z3-backed properties: on variables and authored data_constraint: blocks actually proved. It does not rewrite the YAML. It does not inspect the Go. It does not run Kind2 on a helper.
Zero Z3-backed variables is a pass: no variables with Z3-backed properties or constraints to verify. That pass is silence, not a proof. Z3 missing when claims exist is a warning: install Z3, then rerun. A real solver error with Z3 present is fail. A violated subject is warning, with the variable or requirement, the check name, and the counterexample. An unverified remainder (not proved, not violated) is also warning.
The expensive miss is a merge that tests sampled and a CI runner that never called Z3. The audit then reports that every property was fine, because it only counted the subjects it proved, or because it counted none. This hop is the gate that says the authored subjects closed, or names the ones that did not.
This is not whether a data_constraint was even lowered. That hop is
data constraint Z3 coverage.
A skip list can still pass while a kept property is violated. The proof result is this hop. The help file teaches that first.
# firewall.req.yaml
variables:
- name: allowed_hosts
type: string
direction: output
properties:
merge: union
deduplicate: true
commutative: true
idempotent: true
# last CI: no Z3 on the runner
# proof audit --check z3_properties_verified
# no variables with Z3-backed properties or constraints to verify
# silent pass: the merge was never a subject
The fix is on the property, the fixture, or the variable model, not on another sample. Run proof verify-properties specs/system (the command this hop wraps). Install Z3 if the hop says it is missing (brew install z3, or drop --no-download only on an air-gapped runner that already has the binary). Then re-run VERIFY. Do not raise coverage on the samples to hide a counterexample. The default warning does not block advancement; a quiet hop after you ignore the warning is still not a proof.
proof audit --check z3_properties_verified --verbose
proof verify-properties specs/system
proof help z3_properties_verified
02 · The exhibit
One last green VERIFY on firewall. The tests sampled two hosts. Z3 never saw the merge. Click the tabs.
The stamp
This hop
Nobody asked whether the authored merge proved. A clean sample stamp is not a Z3 subject. The finding kind is this hop.
Need unreadThe stamp
Keep the live tests. Keep the Jama cell. That is not this hop.
Keep the recordProof
Same merge algebra. Silent empty pass, or this hop. Click the tabs.
| Surface | What they do | What Proof does | What we lose |
|---|---|---|---|
| Green tests | The last samples that still unioned. | Whether the authored merge proved for every input Z3 was given. | We do not rerun the suite here. A green stamp is not this hop. |
| Data constraint Z3 coverage | Whether an authored case was even lowered. | Whether the subjects that were lowered actually proved. | Not the skip list. See data constraint Z3 coverage. |
| Data constraints complete | Completeness and exclusivity of the kept cases. | The whole Z3 subject set, including properties: merge algebra. |
Not the partition. See data constraints complete. |
| Behavioral implications verified | Whether a lowered implication holds on the data domain. | Whether the property and constraint subjects closed. | Not the implication class. See behavioral implications verified. |
| Z3 on one function | A lemma hung on a helper. | Variable-level SMT on authored YAML. | Not the lemma. See Z3/Kind2 on one function. |
| Jama cell | A shall, and a parent if you type it. | A warning the audit can name next to the unproved subject. | Not Jama's V&V. Jama still authors. We have not run a frozen Jama pack. |
The teaching graph is still one output whose tests sampled two hosts while Z3 never saw the merge. Close it by proving the authored properties, or by deleting the block if it was never a claim. Do not add another sample only to silence the checker. With no proved subject the empty pass still prints, and the merge did not go anywhere. It just never sat in Z3.
# Z3 is on the runner now
# proof verify-properties specs/system firewall
# firewall: 4 properties proved (max 0.4s)
# proof audit --check z3_properties_verified
# 1 Z3 proof subjects, all 4 proved
#
# VERIFY is allowed to move on. a pass now sits on proved subjects
The skip-list hop stays on data constraint Z3 coverage. The partition hop stays on data constraints complete. The implication hop stays on behavioral implications verified. The helper lemma stays on Z3/Kind2 on one function. Jama still authors. Proof vs Jama.
03 · The honest loss
A quiet proof audit --check z3_properties_verified can still mean there was nothing to prove. Jama still authors.
Warning on a violated subject, on an unverified remainder, and on Z3 missing when claims exist. Fail only when Z3 is present and the verifier itself errors. Zero Z3-backed variables is a pass, not a fail: the hop does not invent properties. The hop does not rewrite the YAML. It does not add a shall. It does not prove the Go. It does not run Kind2. A quiet hop is not a proof that the authored merge is the one you meant, only that every subject Z3 was given closed, or that there were none. It does not count skipped data_constraint rows. That count is the other hop. The default warning does not block advancement. We have not scored this floor against a frozen Jama pack or a ReqIF export. The loss is named, not scored.
The skip-list hop stays on data constraint Z3 coverage. The partition hop stays on data constraints complete. The implication hop stays on behavioral implications verified. The helper lemma stays on Z3/Kind2 on one function. The engagement stays on software correctness audit. Jama still authors.
04 · Nearby questions
What is Z3 properties verified? Same question. Same URL.
Is this data constraint Z3 coverage? No. That hop is whether an authored case was even lowered. This hop is whether the subjects that were lowered actually proved. See data constraint Z3 coverage.
Is this data constraints complete? No. That hop is completeness and exclusivity of the kept cases. This hop is the whole Z3 subject set. See data constraints complete.
Is this behavioral implications verified? No. That hop is whether a lowered implication holds. This hop is whether property and constraint subjects closed. See behavioral implications verified.
Is this Z3 on one function? No. That hop is a lemma on a helper. This hop is variable-level SMT on authored YAML. See Z3/Kind2 on one function.
Does a quiet hop prove the merge is the one you meant? No. The hop observes subjects. It does not score the sentence.
Does a quiet hop prove the code matches the shall? No. The hop observes Z3 subjects. It does not prove the Go.
Does zero Z3-backed variables fail this hop? No. That is a pass. A pass with nothing to prove is not a proof.
Does a violated subject fail the merge? No. The check keeps warning severity unless you raise it in proof.yaml.
Is Proof a Jama alternative for the shall? No. Jama still authors. Proof vs Jama.