Topic · orphan tests

Orphan tests

Gist

Orphan tests run green and consume CI. They do not count as evidence for any live requirement. Proof runs proof lint --check orphan_tests_clean. A coverage percentage is not that check. Jama still authors.

proof lint --check orphan_tests_clean

Keep the suite if it already exercises the function. Keep Jama if it already holds the shalls. Neither one fails the merge when a Test* has no // Verifies: line, or still names a retired id.

Request demo Last green suite vs this stamp

01 · The test nobody owns

A green Test* is not evidence until it names a live requirement.

You can add tests all week. coverage_met still reports the new shall covered by a file that never pointed at it.

Under source_native (the default), every Test* and Fuzz* needs a // Verifies: <REQ> annotation, or the triple form, pointing at a live requirement. The check also catches the stale-id case: a refactor retires OLD-REQ, the test file still says Verifies: OLD-REQ, and the new requirement inherits none of that run.

An orphan test is the inverse of orphan code. The function exists, the suite is green, CI time is spent, and the requirement's evidence count does not move. Go Benchmark* and Example* are not orphans by default. They are scaffolding unless they opt in with an explicit Verifies:. A test bound to a KnownIssue (// Verifies: KI:…, // Reproduces:) is not an orphan either. It proves a bug still exists. Attaching a product requirement to that file would credit a shall the test does not satisfy.

Coverage percentages live on coverage threshold. Whether every shall has some test at all lives on every requirement covered. This hop is ownership of the test symbol, not a line-rate and not a missing SYS-REQ id.

proof lint --check orphan_tests_clean
proof lint --check orphan_tests_clean pkg/policy/
proof audit --scope baseline --verbose

Inspect before you annotate. Four legitimate resolutions, one per finding: write // Verifies: to the requirement the test actually exercises; write the requirement first when nothing states that behavior; relink or delete when the shall was retired; or exclude the subtree with lint.exclude when it is genuinely out of scope. Do not point the nearest plausible id at a helper that does not exercise it. That passes the audit while inflating a shall the test does not support.

02 · The exhibit

Same suite. A green run, or this stamp.

The policy test still passes. Click the tabs.

The suite

Ask did TestApplyAllowsValidPolicy pass
Out ok, coverage 91%
Why no Verifies line, so no shall owns the run

Stamp green

This hop

No orphan_tests_clean JSON. The test is not in any requirement's evidence count.

No stamp

The suite

Keep the test. It still exercises Apply. That is not this hop.

Keep the suite

Proof

Ask does this Test* name a live requirement
Out pkg/policy/apply_test.go:TestApplyAllowsValidPolicy no // Verifies:

Orphan test is named

Same suite. A green run, or this stamp. Click the tabs.

Surface	What they do	What Proof does	What we lose
go test / coverage report	The function ran. The percentage moved.	Fail the hop when the Test* has no live `Verifies:`.	We do not write the annotation. A green stamp is not a proof of the Go.
Dead code / CWE-561	Unreachable functions. go.dev `deadcode`.	Unowned tests, not unreachable code.	We do not find dead functions. That hop is a different instrument.
Every requirement covered	Whether each shall has some test.	Whether this test names a shall.	Not that hop. See every requirement covered.
Jama	The authoring programme. Reviews if you put them there.	Warn when a test is not in the evidence graph.	Not Jama's V&V. Jama still authors. We have not run a frozen Jama pack.

The teaching graph is still one test next to one requirement. Run the check. Read the path and symbol. Annotate, author, relink, or exclude. Do not bulk-stamp Verifies: onto helpers to clear the warning.

// Verifies: SYS-REQ-403
func TestApplyAllowsValidPolicy(t *testing.T) {
    // ...
}

The coverage hop stays on coverage threshold. The shall-has-a-test hop stays on every requirement covered. Jama still authors. Proof vs Jama.

03 · The honest loss

Proof names the orphan. It does not write the requirement.

A green orphan_tests_clean is a warning count of zero. It is not a proof of the Go. Jama still authors.

The check is a WARNING by default, not an error. Under overlay_audit both orphan checks pass wholesale, because upstream-owned tests are not required to carry in-source comments. orphan_scope: traced can vacuous-pass for breadth when unannotated files are skipped. Helper files with no Test* are not flagged. We do not detect unreachable functions, Spring or npm import cycles, or dryer lint. We have not scored this graph against a frozen Jama pack or a second human's test review. The loss is named, not scored.

The coverage hop stays on coverage threshold. The engagement stays on software correctness audit. Jama still authors.

Request demo How the install is bought

04 · Nearby questions

What people type next.

What are orphan tests? Same question. Same URL.

Is this dead code? No. Dead code is unreachable functions (CWE-561, go.dev deadcode). This hop is a live test with no live requirement. We do not find dead functions.

Is this every requirement covered? No. That hop asks whether each shall has some test. This hop asks whether this test names a shall. See every requirement covered.

Is this a coverage threshold? No. That hop is a percentage on the traced set. An orphan never enters that denominator. See coverage threshold.

Is Proof a Jama alternative for the tests? No. Jama still authors. Proof vs Jama.