Postmortem · jsonparser · July 2026
100% MC/DC. Two bugs still escaped.
We had formal requirements, full condition coverage, and a green gate. Two real defects still got through. This is the proof gap, the fix, and what changed afterward.
Looking for the complete requirement and evidence dossier? Open the jsonparser audit.
01 · The subject
Code we could not hide from.
An audit practice should start on code it cannot hide from. We picked jsonparser: a Go JSON parser maintained by Proof’s founder for ten years, 5,600+ stars as of August 2026, with years of issues, community fixes and continuous fuzzing already behind it. Code we had every reason to trust — which is exactly the code that teaches you what trust is worth.
The stakes were not hypothetical. jsonparser sits in dependency graphs it never asked to join: log pipelines, service meshes, API infrastructure. It ships inside Grafana Loki and, transitively, a good slice of the cloud-native stack. A silent bug in a library like this becomes somebody’s incident, three layers up, with a green build.
One caveat belongs at the top of the story. We chose this subject, and we maintain it. Recovering intent from code you already know is the easiest version of the job, and a self-selected subject is a weaker claim than an assigned one. That limitation is published with the rest.
02 · What we believed
What we thought we had proven.
The behavior of the library was formalized into 123 requirements, and code is judged only against the ones a maintainer has approved. Condition-level coverage (MC/DC) measured 100% of decisions and 100% of conditions. Over it all, the strict audit gate: zero errors, zero warnings, on every commit.
That is a strong position, and we held it. All 279 non-test functions declare the requirements they serve in the source itself — most on the function, a handful at the top of their file — so the trace moves when the code moves. That count was taken when the audit closed; master carries 280 today, and git grep -h '^func ' -- '*.go' ':!*_test.go' re-counts it. A structure-aware fuzzer that mutates the key path as well as the bytes was built mid-campaign and stays in the suite.
One calibration story shows why the position felt solid. Years ago, OSS-Fuzz caught a real panic in Delete by mutating bytes at random. Encoded as a solver constraint against the formalized spec, the same defect fell out as a counterexample in six milliseconds. Random search found it once, by luck; once specified, it cannot be missed again.
Everything in that account depends on the behavior being specified first.
03 · What the review found
Six defects the review found.
The count is worth stating carefully, because the artifacts are public and you can count them yourself. proof/problem-reports/ on master holds eight defect files, every one fixed. Six of the eight the review found by itself. The other two were reported by outside contributors and credited by name in the CHANGELOG’s acknowledgments, and those two are the escapes in the next section.
One register record is worth pulling out now, because the next section turns on it — and it sits beside the six rather than among them. KI-1 is a single record standing for a panic class repeated at several call sites: the same unchecked dereference, spelled slightly differently each time. Its only defect file is DEFECT-260726-QS2V, which is one of the two an outside contributor is credited for. A hazard sweep hunted that class, found seven sites, fixed all seven and reported clean. The eighth spelling belongs to the next section.
The review also caught the benchmark lying: since 2017, the suite had been measuring a different library under encoding/json’s name, and the issue reporting it had been open for nine years. And it made the parser faster: formalizing string handling exposed an unbounded backslash scan, and the fix took one hot path from 128 microseconds to 22.
Alongside the review, the decade-old backlog went to zero: fifty open issues and twelve open pull requests, closed or merged across six releases with zero breaking changes. The CHANGELOG carries the release-by-release record, and the ten-year writeup tells it long-form.
The register, read off master →One resolved defect record, in full →
04 · What escaped
Then two got past.
The postmortem on this project is called Two Bugs That Escaped L3 Strict Proof Review, and two is the honest number. Both are in Set(), and both predate the audit by years — writing them was not the failure. Certifying around them was.
One of two. Ask Set to write 9 at index [5] of the array [1,2], and it handed back [9]. No panic. Valid JSON. The rest of the data, gone. The branch that should have grown the array only fired when its first element was an object; scalar arrays fell through to code that overwrote them, and a comment in the source described the overwrite as intentional.
Two of two. Ask Set to write at the path "a" then "", and it panicked. The empty path component was the same failure class the hazard sweep had already closed: seven sites found, seven fixed, reported clean. The eighth was written keys[depth:][0][0] instead of keys[i][0], and the pattern the sweep matched on did not see it.
Fig. 01 · ILLUSTRATIVE EXAMPLE A drawing of a real event; the records behind it sit in the repository. The sweep matched one spelling of an unsafe idiom. The property behind it went unmatched. Seven sites fixed; the eighth escaped and is recorded as a miss.
How they surfaced is recorded honestly, because the record is in tension with itself: our own problem reports file both as audit findings, while the project’s changelog credits an outside reporter for each of the two, by name, in the release that shipped the fixes. The gate stopped neither — that part is not in dispute — and we published the disagreement. The changelog also records three more escapes fixed in v1.3.1 with no postmortem of their own; the audit page records that as a gap in the register, which is a different thing from having had no other misses.
05 · The denominator
The missing denominator.
The affected code carried 100% MC/DC when both defects were live, and the postmortem does not soften what that means: “MC/DC has no notion of ‘correct’; it has only ‘exercised.’” Under it sits the sentence that explains both escapes: “an unspecified partition is … not in any check’s denominator.”
Nobody had written the requirement for what scalar arrays deserve from Set, so no check on earth was counting that case. The coverage number was complete and it was measured over the behavior somebody had named.
The panic site failed the same way for a second reason. The sweep was looking for one spelling of an unsafe construct, and the property behind that construct was never written down, so a slice expression and a second index level were enough to hide the same mistake in plain sight. A pattern can only match what it was given. A requirement is what makes the property itself checkable.
Fig. 02 · ILLUSTRATIVE EXAMPLE A drawing of a real event; the records behind it sit in the repository. Coverage counts the code inside the boundary somebody wrote down. Both defects were outside it, and a complete number said nothing about that.
A gate can only check what a requirement names. Everything else sits outside the denominator, and no coverage figure will tell you it is there.
06 · What changed after
Every escape changed the graph.
We published the root-cause analysis the way we would demand it from a client’s vendor: blameless, specific and public, on the project’s master branch, next to the code it indicts.
Then the hardening. Two requirements that did not exist before, both now approved: SYS-REQ-110 says what Set owes an array index past the end of the array, for every element type, including the ones the old code skipped; SYS-REQ-111 says the empty path component is a defined input on every entry point and must never panic.
And three pieces of machinery, because a requirement with nothing checking it is a sentence: a reference oracle that compares output against the standard library, a fuzzer that mutates the key path as well as the bytes, and a signal rule that flags the unguarded dereference in every spelling, including the one the sweep walked past.
The misses got more than fixes: each one retired as a class. And the class claim stays honest — one passing reproducer proves an instance dead; closing a whole class takes more evidence than one test.
Every new requirement and instrument, with its file →Why the graph gets stronger →
07 · What stayed
The corpus is still in the tree.
Everything this site describes in the abstract exists concretely in that repository, laid out the way a client corpus is laid out: how requirements are stored, how code and tests declare what they answer for, and where the defect history lives.
PUBLIC AUDIT EVIDENCE jsonparser · buger/jsonparser on master. Every path below opens the real file.
specs/stakeholder/requirements/ ← the 7 promises the library makes (STK-REQ-…) specs/system/requirements/ ← the 116 testable clauses (SYS-REQ-…), plain YAML parser.go ← functions annotated with the requirements they implement set_spec_test.go ← // Verifies: SYS-REQ-009 and // Reproduces: KI-3, on one screen proof/known-issues/ ← the register: 4 records, all fixed, each with its command proof/problem-reports/ ← 8 defect records: root cause named, hardening recorded proof/signals/rules/ ← the unsafe pattern from the postmortem, generalized docs/proof-gap-root-cause.md ← the postmortem, on master, where you can re-run both misses
Those counts are read off master. The mechanics of the record are walked file by file on the audit page: one requirement chain end to end, the register read off master, and one resolved defect record in full.
Browse the live graph ↗ SEEDED PRODUCT DEMO opens jsonparser-demo, the seeded fork branch; read the labels first
08 · Re-run it
Nothing here is taken on faith.
Every claim above resolves to one of these four artifacts, and a person validated every finding behind them before it was published.
The postmortem
PUBLIC AUDIT EVIDENCE jsonparser
The blameless root-cause analysis of both misses, on the public master branch. Its own first line will tell you the count is two.
The two defect records
PUBLIC AUDIT EVIDENCE jsonparser
DEFECT-260727-WWWY and DEFECT-260726-QS2V, filed beside the six the review found, each with its reproducer. WWWY is the record of the first escape. QS2V is the record of the sweep the second one got past.
The CHANGELOG
PUBLIC AUDIT EVIDENCE jsonparser
The release that shipped both fixes, and the outside credits that put our own record in tension with it.
The ten-year writeup
Our own narrative, not an audit record. It is commentary on the three artifacts above, and it carries no provenance badge because it is not evidence.
The long-form account of the whole campaign, backlog sweep included. The three artifacts above are the records it draws on.
The audit behind this postmortem is the same engagement we sell: requirements your engineers approve, the code held to them, the evidence in your repository, re-run on every release. Yours stays private, always — the trust page explains why that is structural. A person validates every finding before it reaches you. When a defect slips through, the miss is recorded in your register and the bar is widened. Misses from public Proof work are published, and private engagement evidence stays private unless you authorize disclosure.