The jsonparser case · public audit · July 2026
Ten years of jsonparser, and the two bugs that got past us.
We took the audit we sell and ran it, in public, on a library this practice had maintained for a decade.What the review found, what escaped anyway, and what stayed. Every claim resolves to a public artifact.
Browse the live graph ↗ seeded showcase branch; read the labels first
01 · The subject
Code we could not hide from.
An audit practice should start on code it cannot hide from. We picked jsonparser: a Go JSON parser maintained inside the practice for ten years, 5,500+ stars, with years of issues, community fixes and continuous fuzzing already behind it. Code we had every reason to trust — which is exactly the code that teaches you what trust is worth.
The stakes were not hypothetical. jsonparser sits in dependency graphs it never asked to join: log pipelines, service meshes, API infrastructure. It ships inside Grafana Loki and, transitively, a good slice of the cloud-native stack. A silent bug in a library like this becomes somebody’s incident, three layers up, with a green build.
One caveat before the story, because it belongs at the top rather than in a footnote: we chose this subject, and we maintain it. Recovering intent from code you already know is the easiest version of the job, and a self-selected subject is a weaker claim than an assigned one. That limitation is published with the rest.
02 · The bar, applied
What formalizing ten-year-old code looked like.
The behavior of the library was formalized into 123 requirements: 7 stakeholder promises and 116 system clauses, plain YAML in the tree, each approved before any code was judged against it.
The system clauses carry machine-checkable formal specifications — FRETish compiled to temporal logic. All 279 functions are annotated with the requirements they serve, in the source itself, so the trace moves when the code moves. Condition-level coverage (MC/DC) was measured at 100% of decisions and 100% of conditions. A structure-aware fuzzer that mutates the key path, not only the bytes, was built mid-campaign and stays in the suite. Over it all, the strict audit gate: zero errors, zero warnings, on every commit.
One calibration story. Years ago, OSS-Fuzz caught a real panic in Delete by mutating bytes at random. Encoded as a solver constraint against the formalized spec, the same defect fell out as a counterexample in six milliseconds. Random search found it once, by luck; once specified, it cannot be missed again.
The bar itself is published →Walk one requirement end to end →
03 · What the review found
Six defects, and a decade of backlog.
The count is worth stating carefully, because the artifacts are public and you can count them yourself. proof/problem-reports/ on master holds eight defect files, every one fixed. Six of the eight the review found by itself. The other two were reported by outside contributors and are credited by name in the CHANGELOG’s acknowledgments — and those two are the two published misses in the next section.
Beside the defect files sits the register: four known-issue records, all fixed. Four records for six defects is not a discrepancy — KI-1 is one record standing for a panic class repeated at several call sites: the same unchecked dereference, spelled slightly differently each time. A hazard sweep hunted that class, found seven sites, fixed all seven and reported clean. What happened to the eighth spelling belongs to the next section.
The review also caught the benchmark lying: since 2017, the suite had been measuring a different library under encoding/json’s name, and the issue reporting it had been open for nine years. And it made the parser faster, not slower: formalizing string handling exposed an unbounded backslash scan, and the fix took one hot path from 128 microseconds to 22.
Alongside the review, the decade-old backlog went to zero: fifty open issues and twelve open pull requests, closed or merged across six releases with zero breaking changes. The CHANGELOG carries the release-by-release record, and the ten-year writeup tells it long-form.
04 · What escaped
Then two got past.
The postmortem on this project is called Two Bugs That Escaped L3 Strict Proof Review, and two is the honest number. Both are in Set(), and both predate the audit by years — writing them was not the failure. Certifying around them was.
One of two. Ask Set to write 9 at index [5] of the array [1,2], and it handed back [9]. No panic. Valid JSON. The rest of the data, gone. The branch that should have grown the array only fired when its first element was an object; scalar arrays fell through to code that overwrote them, and a comment in the source described the overwrite as intentional. Nobody had written the requirement for what scalar arrays deserve, so no check on earth was counting it.
Two of two. Ask Set to write at the path "a" then "", and it panicked. The empty path component was the same failure class the hazard sweep had already closed: seven sites found, seven fixed, reported clean. The eighth was written keys[depth:][0][0] instead of keys[i][0], and the pattern the sweep matched on did not see it.
Fig. 01 · The sweep matched one spelling of an unsafe idiom, not the property behind it. Seven sites fixed; the eighth escaped and is published as a miss.
The affected code carried 100% MC/DC at the time, and the postmortem does not soften what that means: “MC/DC has no notion of ‘correct’; it has only ‘exercised.’” And the deeper sentence under it: “an unspecified partition is simply not in any check’s denominator.” Nobody had named scalar arrays or the empty path component, so no check was counting either.
How they surfaced is recorded honestly, because the record is in tension with itself: our own problem reports file both as audit findings, while the project’s changelog credits an outside reporter for each of the two, by name, in the release that shipped the fixes. The gate stopped neither — that part is not in dispute — and we published the disagreement rather than quietly keeping the account that flatters us. The changelog also records three more escapes fixed in v1.3.1 with no postmortem of their own; the ledger lists that as a gap in the register, not an absence of misses.
05 · What changed after
Every escape became a gate.
We published the root-cause analysis the way we would demand it from a client’s vendor: blameless, specific and public, on the project’s master branch, next to the code it indicts.
Then the hardening. Two requirements that did not exist before, both now approved: SYS-REQ-110 says what Set owes an array index past the end of the array, for every element type rather than only the one the old code handled; SYS-REQ-111 says the empty path component is a defined input on every entry point and must never panic.
And three pieces of machinery, because a requirement with nothing checking it is a sentence: a reference oracle that compares output against the standard library, a fuzzer that mutates the key path rather than only the bytes, and a signal rule that flags the unguarded dereference in every spelling, including the one the sweep walked past.
The misses got more than fixes: each one retired as a class. And the class claim stays honest — one passing reproducer proves an instance dead; closing a whole class takes more evidence than one test.
06 · What stayed
The corpus, still in the tree.
Everything this site describes in the abstract exists concretely in that repository, laid out the way a client corpus is laid out: how requirements are stored, how code and tests declare what they answer for, and where the defect history lives.
specs/stakeholder/requirements/ ← the 7 promises the library makes (STK-REQ-…) specs/system/requirements/ ← the 116 testable clauses (SYS-REQ-…), plain YAML parser.go ← functions annotated with the requirements they implement set_spec_test.go ← // Verifies: SYS-REQ-009 and // Reproduces: KI-3, on one screen proof/known-issues/ ← the register: 4 records, all fixed, each with its command proof/problem-reports/ ← 8 defect records: root cause named, hardening recorded proof/signals/rules/ ← the unsafe pattern from the postmortem, generalized docs/proof-gap-root-cause.md ← the postmortem, on master, where you can re-run both misses
The mechanics of that record are walked, file by file, on the jsonparser audit page: one requirement chain end to end, the register read off master, and the postmortem in full.
07 · Re-run it
Nothing here is taken on faith.
Every claim above resolves to one of these four artifacts, and a person validated every finding behind them before it was published.
The postmortem
The blameless root-cause analysis of both misses, on the public master branch. Its own first line will tell you the count is two.
The 123 requirements
The full formalized corpus: 7 stakeholder promises, 116 system clauses, plain YAML.
The repository
Specs, proof artifacts, reproducers and the audit gate, in the tree where they live.
Walk one finding end to end →The register, read off master →The second public audit: rsync →
08 · On your code
This, pointed at your component.
The audit above is the same engagement we sell: requirements your engineers approve, the code held to them, the evidence in your repo, re-run on every release. The difference is that yours stays private, always — the trust page explains why that is structural, not preference.
A person validates every finding before it reaches you. And the misses, when they come, get the same treatment you just read.