Topics / Engineering practice
Measure the evidence behind software quality metrics.
A number becomes useful when its denominator, scope, revision and decision are clear. Counts of tests, requirements or findings are not quality on their own.
Give every metric a question to answer.
| Metric | Useful question | Common trap |
|---|---|---|
| Requirement evidence coverage | Which in-scope promises have observed support? | Counting a linked but unexecuted test as verified |
| Structural coverage | Which code or decisions were exercised? | Treating execution as correct behavior |
| Open findings | What unresolved work affects this change? | Rewarding volume without validation |
| Review effort | Where does acceptance consume human time? | Ignoring preparation and later rework |
Keep the denominator visible.
Illustrative example
Coverage improves because the scope shrank.
A report rises from 80% to 95% after difficult requirements are excluded. That may be a legitimate scope decision, but the percentage alone hides it. Retain the included set, exclusions, authority and reason so the comparison can be interpreted.
For a trend, compare like with like. Changes in product risk, team mix or test environment can matter more than the headline movement. Preserve missing data as missing.
Measure Proof as a workflow, too.
Track the active human effort needed to accept a change, the work reused from earlier records and later regressions. Include onboarding and evidence maintenance. A larger graph is useful only when its records help a real decision.
Proof can connect requirements to recorded evidence; the validity of a metric still depends on the underlying links and executions. Review a sample of the records behind the number before relying on it.
Try it on your software
Choose a quality metric that no longer helps decisions.
We can inspect its underlying evidence and define a more useful pilot measurement.