Topic · Correctness vs security
We passed a security audit but still ship functional bugs. What kind of audit catches those?
Proof installs a correctness audit on one component. Your engineers approve the shalls. Then proof audit fails the build when a retry captures twice, even if Snyk, the pentest, and SOC 2 are green.
proof auditKeep the scanner. Keep the pentest. Playwright, Sentry, and BrowserStack keep the happy path. None of them re-read the code against the promise you sold.
01 · The distinction
A CVE-clean scan does not know the shall.
Security work asks whether someone can steal the session. Correctness work asks whether the component still does what you promised when nobody is attacking it.
A capture of forty dollars times out. The client retries. The network records two captures. No CVE named that. No OWASP item named that. The pentest scope did not include “do not capture twice.” Snyk is still green. The customer is not.
That is not a missed vulnerability. It is a missed obligation. The bar has to be a shall the owners signed, held to the code, on every commit. Crowdtesting can click Pay once. Contract tests can agree on a JSON shape. Neither of those is the shall.
proof audit --check spec_lint_ac_subset_of_satisfies
proof audit --check acceptance_criteria_witnessedspec_lint_ac_subset_of_satisfies fails if an acceptance criterion is not a subset of a shall. acceptance_criteria_witnessed fails if a shall has no witnessed criterion. A SOC 2 letter does not have those exits. Neither does a pentest PDF.
02 · The exhibit
Four greens. One broken shall.
Shall: after a capture succeeds, a retry must not capture again. The models that searched this question named Playwright, Sentry, LaunchDarkly, BrowserStack, Pact, Percy, and TestRail. Those are the right tools for their jobs. Here is the split.
| What ran | Green? | What it looked at |
|---|---|---|
| Pentest / SAST / Snyk | Yes | Injection, secrets, known CVEs. Not the retry. |
| SOC 2 Type II | Yes | Access control and change tickets. Not the shall. |
| Playwright happy path | Yes | Click Pay once. The timeout path was not in the script. |
proof audit |
No | The approved shall on retry. Finding with a reproducer. |
If the job is “can we be hacked,” Cobalt is the right firm. If the job is “did we follow the controls we wrote,” Schellman is the right firm. If the job is “this capture must not fire twice,” those firms are the wrong instrument. That is the audit we run.
03 · The honest loss
This is not a pentest, not SOC 2, and not a CVE inbox.
Proof will not produce a penetration-test report, a Type II letter, or a vulnerability feed. Keep those vendors.
We have not run a pentest and a correctness audit on the same frozen corpus. A green proof audit means the checks we selected passed on the approved shalls. It does not mean the attack surface is clean, and it does not mean we selected every promise a salesperson ever made.
Snyk still owns the scanner. The named split is on the Snyk compare. Jama still wins at programme authoring. The commercial shape of the install is on the audit page. The independent-check seed lives on software correctness audit.
04 · Nearby questions
What people type next.
Penetration testing checks if we can be hacked. What checks if the software actually works correctly? A pentest is the attack surface. Proof re-reads the shalls. Keep Cobalt. Run proof audit on the component you sold.
Is Proof a Snyk alternative? No. Keep the scanner. A refund that exceeds capture is not a vulnerability. That page is Proof vs Snyk.
How do I get an independent check that the software does what we promised? One component, approved shalls, the same gate in CI. That seed is on the independent-check page.