Topic · Coverage
What's the difference between test coverage and requirements coverage?
Test coverage asks whether a line ran. Requirements coverage asks whether an approved shall still has a witness at its own level. Proof fails the build on the second.
proof audit --check acceptance_criteria_witnessedgo test -cover still owns the first job. A 100% line report can be all unit tests on children. It never asked whether the shall itself was witnessed.
01 · The distinction
The denominator is the whole argument.
A coverage number is a fraction. Change the bottom of the fraction and the same suite tells a different lie.
ISO 26262 Part 6 keeps the two jobs in different tables. Table 9 is requirements-based testing: highly recommended at every ASIL. Table 10 is structural coverage: statement, then branch, then MC/DC, scaling with ASIL. Those are not synonyms. NASA NPR 7150.2D and DO-178C make the same split. The certificate cluster for that discipline lives on the DO-178C page, not here.
Proof's requirements-coverage check is not a second line counter. A stakeholder criterion marked testable needs an acceptance test of its own, annotated to that criterion. A unit test on a child does not count. That is the V-model row, not a style lint.
proof audit --check acceptance_criteria_witnessed
proof audit --check spec_lint_ac_subset_of_satisfies
proof mcdc measure ./... --engine goThe first command is red when a testable shall has no acceptance witness (or the miss is staged as counted warning debt). The second is red when an acceptance citation and the child's satisfies back-link disagree. The third is structural coverage on Go. Three jobs. Three exit codes.
02 · The exhibit
Same suite. Two bottoms of the fraction.
A green line report can sit next to a red shall. That is not a tool bug. It is two denominators.
| Question | Test coverage | Requirements coverage |
|---|---|---|
| Denominator | Lines, branches, or conditions that exist in the file. | Shalls an owner approved. |
| Green means | This code ran under some test. | This shall has a witness at its own level. |
| ISO 26262 column | Part 6 Table 10 (statement / branch / MC/DC). | Part 6 Table 9 (requirements-based testing, ++ at every ASIL). |
| Typical CI | go test -cover, Istanbul, JaCoCo. |
proof audit --check acceptance_criteria_witnessed |
| 100% still misses | A promise nobody exercised as that promise. | A promise nobody wrote down as a shall. |
| Condition-level | MC/DC on the decision. Different page. | Not a substitute for MC/DC. Not a line report either. |
Drawn for this page from the standard's split, not from a score we ran on someone else's analyzer. Fault injection and back-to-back testing are also Table 9/10 neighbours. Proof does not inject faults. A rewrite that must match old behaviour is the mirrors page.
# test coverage: the serve-from-cache line ran go test -cover ./... # requirements coverage: the shall itself has an acceptance witness proof audit --check acceptance_criteria_witnessed
If the shall is “imports 100,000 records,” a unit test that covers the parser loop does not witness it. The acceptance test that actually imports 100,000 records does. The coverage report can still print 100% either way.
03 · The honest loss
This is not a second coverage percentage, and it is not ISO 26262.
Keep go test -cover. Keep the ASIL safety case if you certify a vehicle. Proof will not produce those artefacts.
We do not replace Istanbul or JaCoCo. We do not instrument C. Structural MC/DC on Go is a different URL: MC/DC for Go. Condition-level coverage is that page, not a twin of this one.
A green acceptance_criteria_witnessed means the testable shalls in the graph have an acceptance witness, or the miss is staged as counted debt. It does not mean every promise a salesperson made is in the graph. Claims that never became a shall are outside the bar until an owner writes them and signs them.
Jama still wins at programme authoring. ISO 26262 Part 6 as a certificate, SOTIF triggering conditions, and a TÜV functional-safety assessment are outside Proof. Those standards sit as a mention on the DO-178C discipline page, not as a second standards URL.
04 · Nearby questions
What people type next.
How do I get condition-level coverage, not just line coverage? That is MC/DC: each condition shown to flip the outcome on its own. Measure it on Go. VectorCAST still wins on qualified C.
We have high test coverage but still get escaped defects. The suite never pinned the shall. Recurring bugs are the why-bugs page.
What is a requirements traceability matrix? The matrix stores asserted links. Coverage of those links is this page. The ageing database is the RTM page.