Methodology

The bar we audit against

Proof audits whether software does what was promised, against a published bar. Six clauses. Each one enforced by a machine check, each one anchored in practice older than us. “Verified” should be a claim you can re-run; this page is what we mean when we say it.

Page job: a hostile staff engineer, sent by their CTO to vet the vendor, reads this and reports back “not slop”. ⚠ DECISION (binding): this page never calls itself a “standard” anywhere in visitor copy. A self-crowned standard with zero external adopters is the top-comment kill shot on any forum thread. Framing is “our published bar”, self-imposed, with every clause anchored to external canon (FRET, DO-178C, ISO 26262, disclosure norms) so it reads as operationalizing forty years of established practice rather than inventing authority. Grep the visitor copy for “standard” before ship; it should not appear.

§ 1 · Preamble

Why “verified” needs a definition

When paperwork can be generated faster than it can be read, the artifact stops being evidence, and the word “verified” carries no information unless whoever says it publishes what it means. Here is what we mean. A claim is verified when you can re-run it: the requirement it traces to, the check that enforces it, and the reproducer that fails without the fix, all executable by you, on your tree, without us in the room.

Scope note: this is the bar we hold ourselves to, not a certification scheme. Nobody accredits it and nobody is asked to adopt it. It is published so you can hold us to it.

Job: define the moment without naming anyone, then pre-empt the “self-appointed authority” objection in the same breath the reader would raise it. ⚠ Off-page rule: the spring collapse stays an unnamed allusion in all copy. The Delve reference stays off-page entirely; it exists for sales conversations, and only if the prospect raises it first.

§ 2 · The clauses

Six clauses, each with the check that enforces it

A clause without an enforcement mechanism is a preference. Each clause below names the machine check that fails when the clause is broken, and the external practice the clause is drawn from. The instruments behind the checks are their own page.

1 · Requirements are proven before code is judged

CLAUSEEvery requirement in scope is written in structured English with formal semantics, then proven realizable, consistent and non-vacuous before any code is judged against it. A spec that contradicts itself, or that a do-nothing system could satisfy, is a defect in its own right, and it is found first.

THE CHECKRealizability, consistency and vacuity are discharged at spec time by the Kind2 model checker and the Z3 solver. A requirement that fails them never enters the audit.

THE ANCHORNASA's FRET program, which gave structured English requirements machine-checkable semantics. In FRET's published research, a realizability check caught an eVTOL spec defect that permitted backwards flight, in 14 seconds.

2 · No reproducer, no finding

CLAUSEEvery finding ships with a runnable reproducer that fails on the scoped tree, or it does not ship. A finding you cannot re-run is an opinion with formatting.

THE CHECKThe reproducer is executed before delivery and observed red. No red run, no ledger entry.

THE ANCHORCoordinated-disclosure practice in security research: proof of concept before report, so the recipient can verify before they triage.

3 · Traceability at 100% precision

CLAUSEEvery requirement in scope carries links to the code that implements it and the test that verifies it, and every link that exists is machine-checked to resolve and be correct (100% precision).

THE CHECKThe annotation resolver walks every link on every run; a link that fails to resolve fails the gate, and code answering to no requirement is flagged as orphan from the other direction.

THE ANCHORBidirectional traceability, required by DO-178C in avionics and ISO 26262 in automotive, for the same reason it is required here: unlinked evidence cannot be audited.

4 · MC/DC on the decision logic under audit

CLAUSEThe decision logic under audit gets condition-level MC/DC: every condition in every scoped decision is shown to independently affect the outcome, so a test suite cannot look thorough while exercising half the logic. Together with clause 3, this is coverage counted twice: once against the intent, once against the code.

THE CHECKMC/DC instrumentation measures coverage on the scoped decisions. The number is measured, never estimated.

THE ANCHORDO-178C Level A, the coverage bar avionics sets for software whose failure is catastrophic.

5 · A named reviewer signs every finding

CLAUSEA named reviewer signs every finding before it reaches you.

THE CHECKThe gate refuses to deliver a finding without a recorded reviewer identity on the finding record. There is no delivery path that skips it.

THE ANCHORThe oldest norm in professional practice: an audit opinion carries the engagement partner's name, an engineering drawing carries the stamp of the engineer who answers for it.

6 · The gate re-runs, and runs without us

CLAUSEThe audit gate re-runs on every change and runs without us. If the evidence could only be checked in our presence, it would be testimony, and this page promised you evidence.

THE CHECKThe gate runs in your CI, on your infrastructure, with no network dependency on us and no license check. If we disappear, it keeps running.

THE ANCHORThe reproducibility norm in experimental science: a result only the original lab can produce is not yet a result.

Job: the spine of the page. The clause / check / anchor triple is the anti-slop pattern; every claim names the mechanism that fails when the claim is broken, and borrows authority from external canon instead of asserting its own. ⚠ VERIFY before ship: Kind2/Z3 wording and the FRET eVTOL citation must match engine.html exactly; “annotation resolver” must match the product's real check name. The eVTOL fact is this page's only third-party citation; zero survey stats on this page by design (the two-stat budget is deliberately unspent).

§ 3 · Declared limits

What this audit does not see

Assurance here is bounded on purpose: within a declared scope, for declared behaviors, with evidence commensurate with the consequence of failure. The table below is the edge of that boundary, by failure class.

Failure class	Coverage	Note
Logic and intent gaps	Covered	The core of the audit: behavior that violates an approved requirement, or a requirement the team never wrote down.
Boundary and error-path defects	Covered	Condition-level coverage forces the branches ordinary suites skip. Error paths are where most of these hide.
Concurrency interleavings	Partial	Modeled where declared, rarely exhaustive. The report states which interleavings were checked.
Performance under load	Not covered	We make no load, soak or latency claims. A correct system can still be a slow one.
Security exploitation	Complemented, not replaced	We audit whether the software does what was promised. For whether it can be broken into, we point you at the security firms; that is their question.
Third-party dependency internals	Not covered	Dependencies are held to their declared contracts. We do not audit inside them.

Job: the credibility centerpiece for the hostile reader. A vendor that names its blind spots in a table, including two flat “not covered” rows, is doing the thing slop never does. Row order runs covered to not-covered so the eye lands on the admissions last. Light band to mark it as a reference object rather than prose.

§ 4 · Spec provenance

Signed intent: where the spec comes from

The audit starts by reading what you already have. The code first, as the source of truth for what the component actually does. Then its history, for what changed and when. Then the systems where intent actually lives: the issue tracker and the support queue, for what the component promised and where it failed someone. Code knows what it does. It doesn't know why. The why is what we recover.

The recovery matters because requirements drafted only from the code would be circular: they mirror the code's assumptions and grade the code against itself, and every defect already in the code becomes a clause the spec blesses. So the recovered requirements go to the engineers who own the component for approval before any code is judged, which anchors the standard outside both the code and whoever wrote it. On one engagement, an enterprise API gateway, that step produced 47 approved requirements in about two hours of the owners' time. Judged against those requirements, the component surfaced a nil-pointer crash class that every existing test had missed. The tests had been written against the code. The requirements were written against the intent.

Job: pre-empt the objection the staff engineer will raise first (“why can't we just infer the spec from the repo, or have a model do it”). The circularity argument is the load-bearing sentence. Uses only the anonymized enterprise-gateway facts from the inventory (47 requirements, ~2 hours, nil-pointer class). Never name Tyk; “an enterprise API gateway” is the approved phrase.

§ 5 · Severity

Severity, and what happens when we miss

Severity on every finding is our assessment, weighted by reachability and impact; a starting point for your triage, not a CVSS determination.

The ledger also shows the triage boundary. Engagement reports include what we investigated and dismissed, with the reason each candidate was closed, so you can see where the audit stopped looking and judge whether it stopped in the right place.

And when a defect escapes the audit entirely, it is not quietly absorbed. The escape is recorded with its root cause named: which requirement was missing, which test partition, which coverage condition. Closure requires corrective artifacts, and a claim that a whole bug class is fixed is only as strong as the evidence behind it. The ledger's first entry is this protocol applied to our own library; the case is public.

Job: kill the “vendor-inflated severities” objection with provenance instead of protest. The first sentence is the mandated severity-provenance sentence from the brief, used once, verbatim. Publishing dismissals is the differentiator; slop never shows its discard pile.

§ 6 · AI disclosure

How AI is used

Drafting is machine-assisted. Requirements, reproducers and candidate findings are drafted with AI, and nothing is trusted as drafted: deterministic checkers reject invalid work, including requirements that fail realizability, reproducers that never run red, and links that do not resolve. What survives the checkers goes to a named reviewer, who signs the finding or kills it.

We call this posture expert-signed, machine-verified. You will not read a claim here that AI was kept away from the work. The claim is that nothing machine-drafted reaches you unchecked, and that a person with a name signed what did.

Job: disclose without flinching. The hostile reader assumes AI is involved; denying it would burn the whole page, so the disclosure names the exact rejection mechanisms instead. House rule (binding): never claim “we don't use AI”. The posture phrase “expert-signed, machine-verified” is the approved formulation from the brief.

§ 7 · Next

Hold us to it

The bar above is applied inside one kind of engagement. What can be shown publicly accumulates in the ledger; client corpora never do, by design.

The engagement built on this bar Inspect the findings ledger

The ledger includes one miss of our own, with its postmortem.

Job: handoff, no form. The staff engineer on this page is a reporter rather than a buyer; asking for their email here would undercut the whole read. They fork to the ledger for proof or carry the offer page back to the CTO. Conversion belongs to continuous-correctness-audit.html.