Proof. Request demo

Topic · property based testing

What is property based testing, and how do I keep the fixtures true in CI?

Gist

Property based testing checks a general property, not one example. Proof writes the fixtures from the same variable model used by Z3. proof proptest specs/system autopilot --source z3 --output tests/properties/ emits JSON from authored properties: and data_constraint: blocks. The proof itself is proof verify-properties. Hypothesis still samples. Proof does not run those samples.

proof proptest specs/system autopilot --source z3 --output tests/properties/

Keep Hypothesis if you already shrink random cases. Keep Jama if you author the programme. Proof is not a generator library, and a green proptest is not a universal proof.

Request demo Last Hypothesis suite vs this graph

01 · The file that becomes a fixture

A random sample is not this push.

A property is a claim about every legal input. A person can write one example. The next commit does not reopen that example.

The shall-derived case H1 lives on requirements based testing. The lemma H1 lives on Z3 / Kind2 on one function. This page is the fixture: authored variable semantics in, concrete JSON out.

Heuristic source enumerates edge cases from the YAML. Z3 source extracts boundary and counterexample-style values from the model. Variables without properties: or data_constraint: are counted and produce no examples. The JSON is an input for your harness. It is not the assertion, and it is not the SMT result.

proof verify-properties specs/system autopilot
proof proptest specs/system autopilot --output tests/properties/
proof proptest specs/system autopilot --source z3 --output tests/properties/
proof audit --fail-level warn

The first command is the proof. The second is the stable edge list. The third is the model-derived list. Pasting last week's Hypothesis seed into CI is how the property became wallpaper.

02 · The exhibit

Same heading. A sample, or a fixture.

The suite still shrinks random floats. The YAML will take wrap_360 and refuse a silent empty run. Click the tabs.

Hypothesis / QuickCheck

  • Opened float generator, last seed
  • heading one shrunk example that passed
Suite green

This push

The suite did not notice wrap_360 was never authored.

No fixture

Hypothesis / QuickCheck

Still last seed. Still a sample.

Keep the library

Proof

  • Ask does wrap_360 still match this YAML
  • Gate verify-properties, then proptest
No properties: block, no examples

Same heading. A sample, or a fixture. Click the tabs.

Surface What they do What Proof does
Property based testing Random generators, shrinking, a general claim over sampled inputs JSON under tests/properties/ from authored properties: and data_constraint:. Heuristic or Z3 source.
Hypothesis / QuickCheck / Kotest / gopter Run the property against generated values in the language's test runner They still win at running tests. Proof writes fixtures. It does not replace go test.
Requirements based testing Cases from the shall, not from a generator That H1 lives on requirements based testing as proof testgen, not a twin.
Z3 / Kind2 on the function A proof over all inputs of a lemma you wrote The SMT result is proof verify-lemma and proof verify-properties. This URL is the fixture, not the proof.
Mutation testing Kill mutants to see whether the suite notices Not this product. PITest stays with that SERP. The class-of-bugs H1 lives on bugs my tests never check.
Antithesis Deterministic simulation over a whole system Not this product. A variable YAML is not a hypervisor schedule.

Author the semantics first. A heading that wraps at 360 is a variable property, not a hope that the generator hits 359.9.

variables:
  - name: heading
    type: real
    direction: output
    properties:
      normalize: wrap_360
proof verify-properties specs/system autopilot
proof proptest specs/system autopilot --source z3 --output tests/properties/
proof audit --fail-level warn

A thin YAML makes a thin fixture list. Treat the JSON as an input, not as a second source of completeness. We have not run this against a frozen Hypothesis / QuickCheck corpus, and we have not claimed the fixtures prove the Go. The loss is named, not scored.

03 · The honest loss

A green Proof proptest is not Hypothesis.

Proof writes fixtures from authored semantics. It does not sample the type. Jama still authors. Hypothesis still shrinks.

Proof does not implement a random generator, a shrinker, or a language-native @given. Variables without metadata produce no examples. Heuristic fixtures are not MC/DC. Z3 fixtures are optional examples; they are not required by default. The solver proof is owned by verify-properties. Object-code coverage on C still wants the qualified toolchain. That H1 lives on MC/DC for Go and on Proof vs LDRA. Proof generates inputs. It does not run them. Keep go test.

The shall-derived case H1 stays on requirements based testing. The lemma H1 stays on Z3 / Kind2. Jama still authors.

Request demo How the install is bought

04 · Nearby questions

What people type next.

What is property-based testing? The hyphenated spelling. Same URL. Not a twin.

What is property testing? Same cluster, shorter head. Same command.

Does Proof replace Hypothesis? No. Keep the library for shrinking. This URL is the fixture from the YAML.

What is requirements based testing? Cases from the compiled shall, not from a generator. RBT.

How do I prove a specific function meets its specification using something like Z3 or Kind2? The SMT result, not the fixture. Z3 / Kind2.

How do I find the class of bugs my tests never check for? Realizability and Kind2 on the shall. that class.

Is Proof a VectorCAST alternative? No. LDRA owns that cluster. Proof vs LDRA.