Topic · Requirements based testing

What is requirements based testing, and how do I keep the vectors true in CI?

Gist

Requirements based testing derives cases from the shall, not from last week's suite. Proof compiles the FRETish and writes the vectors. proof testgen specs/system autopilot --engine heuristic emits one JSON case per trigger, response, and boundary obligation. Pass --engine kind2 when Kind2 is installed and you want FLIP independence pairs. The audit fails the merge when those fixtures go stale.

proof testgen specs/system autopilot --engine heuristic

Keep Jama if you author the programme. Keep VectorCAST if you need a qualified C generator. Proof writes the inputs. It does not run go test.

Request demo Last suite vs this graph

01 · The suite that forgot the shall

A green suite is not requirements based testing. Last week's cases are not this push.

People write tests from the code they already have. The shall sits in a spreadsheet. The two never meet again.

The coverage H1 lives on test coverage vs requirements coverage. ISO 26262 Part 6 Table 9 is the process name for this cluster. Table 10 is structural coverage. Those are not synonyms. The instrument H1 lives on MC/DC for Go. This page is the generation: vectors from the compiled shall, checked in, stale when the shall moves.

The default engine reads the FRETish structure and writes trigger, response, and boundary traces without a solver. Kind2 and NuSMV rebuild FLIP independence pairs from the same ftExpanded formula the 288-template compiler already produced. That is the point of compiling the shall once. Rebuilding LTL from parsed bits is how coverage used to lie.

proof testgen specs/system autopilot --engine heuristic --output tests/
proof testgen specs/system autopilot --engine kind2
proof testgen specs/system autopilot --check-stale
proof audit --fail-level warn

The first command is the intake that always runs. The second is FLIP when Kind2 is pinned. The third is the stale check. The fourth is the standing gate. Pasting last week's cases into the suite is how the shall became wallpaper.

02 · The exhibit

Same shall. A suite, or vectors.

The spreadsheet still names the requirement. The graph will write the cases from the compiled formula. Click the tabs.

Word pack / last suite

Opened SYS-REQ-001 in the spreadsheet
Cases happy path the author remembered

Suite received

This push

The suite did not notice the shall changed.

No testgen

Word pack / last suite

Still last CCB. Still a file.

Keep the ALM

Proof

Ask does the compiled shall become vectors
Gate testgen, then --check-stale, then audit

Merge blocked if fixtures are stale

Same shall. A suite, or vectors. Click the tabs.

Surface	What they do	What Proof does
Spreadsheet RBT	A person maps each shall to a case they already wrote	Writes JSON from the compiled FRETish. The mapping is the generator, not a cell in Excel.
Heuristic engine	No solver. Structure of the sentence.	Trigger, response, and boundary traces. Default. No Kind2 required.
FLIP / Kind2 / NuSMV	Model checking for independence pairs	`--engine kind2` or `nusmv` on the same `ftExpanded` formula. Positive effect and negative effect per proposition. Kind2 must be installed.
VectorCAST / LDRA	Qualified generators on C and Ada. Object-code coverage.	They still win at the avionics toolchain. That H1 lives on Proof vs LDRA. Proof is not DO-330.
Simulink Test	Model-in-the-loop cases tied to a Simulink requirement	Not supported. A FRETish graph, not a block diagram.
Arithmetic shall	`altitude > 0` as a boolean formula	FLIP cannot express it. On the system compiler spec that was SYS-REQ-001: 18 of 19 requirements got FLIP obligations, 100 cases, 96.2% MC/DC on the boolean rest. The miss is named.

FLIP fixtures are opt-in evidence. The check flip_fixtures_exist is off until you set project.checks.flip_fixtures_exist.enabled: true and list the component. Solver, audit, and document flows do not invent a test_suite_generated result. Only proof testgen does.

proof testgen specs/system autopilot --engine heuristic --output tests/
proof testgen specs/system autopilot --check-stale
proof audit --fail-level warn

A thin shall makes a thin suite. Treat the JSON as a view of the formula, not as a second source of completeness. We have not run this against a frozen VectorCAST or LDRA corpus, and we have not scored FLIP on every arithmetic comparison in the wild. The loss is named, not scored.

03 · The honest loss

A green Proof testgen is not a test runner.

Proof writes the vectors. It does not execute them. Keep go test. Jama still authors. VectorCAST still owns qualified C.

Proof does not generate Gherkin, drive Simulink, or claim a DO-330 qualified tool. Heuristic traces are not MC/DC. Kind2 and NuSMV are optional binaries. Arithmetic comparisons fall out of FLIP. Object-code coverage on C still wants the qualified toolchain. That H1 lives on MC/DC for Go and on Proof vs LDRA.

The coverage-split H1 stays on test coverage vs requirements coverage. The authoring H1 stays on Proof vs Jama. Jama still authors.

Request demo How the install is bought

04 · Nearby questions

What people type next.

What is requirements-based testing? The hyphenated spelling. Same URL. Not a twin.

What is specification based testing? Same cluster, smaller Ads head. Same command.

What is test case generation? The generic head. This URL is the shall-derived case, not an LLM writing tests from the function.

What is ISO 29119? A software testing process standard. Not a generator. Do not mint.

How do I measure MC/DC coverage for my Go code? The instrument, not the generator. MC/DC for Go.

What is the difference between test coverage and requirements coverage? Table 10 versus Table 9. that split.

Is Proof a VectorCAST alternative? No. LDRA owns that cluster. Proof vs LDRA.