Topics / Engineering practice
What should a software correctness audit deliver?
A useful audit investigates whether a bounded component meets its intended behavior. The deliverable is an inspectable case, including what was found, reproduced and left unresolved.
Agree on the scope before collecting findings.
Choose the component, revision, operating conditions and important promises. Identify a person who can resolve ambiguous behavior. A codebase with no authoritative expectations needs requirements recovery as part of the work, not an auditor quietly declaring the current implementation correct.
Include the cost of making the project runnable. Build failures, missing fixtures and inaccessible integrations limit what can be established. Record those limits instead of turning them into a clean result.
Ask for evidence behind each outcome.
| Outcome | Minimum useful explanation |
|---|---|
| Reproduced defect | Expected behavior, failing input, observed result and revision |
| Not reproduced | Attempted conditions and why the allegation remains unresolved |
| Fixed instance | Original reproducer passes on the fixing revision |
| Broader protection | Additional class-level checks and any remaining gaps |
Illustrative example
The crash is fixed, but data is still lost.
A patch prevents a parser panic by returning an empty result. The crash reproducer passes. If the contract requires preserving valid entries, the fix needs another behavioral check. Closing the crash is not enough to close the data-loss risk.
Sample engagement
One component. A handoff your engineers can use.
This sample shows how we would scope a tenant API migration. The observations come from our runnable teaching fixture, not a customer engagement.
Agree the promise and the boundary
Promise: a tenant can update its own document. Another tenant’s request must be rejected without changing stored data.
Inputs: old and candidate revisions, a working build, representative requests and the owner who can settle intended behavior. Start with response status and committed database state. Concurrency, load and live cutover are separate obligations to scope.
Keep these six artifacts
- Intent record: the requirement, its source and approval state, plus behavior the change must preserve.
- Impact and risk map: affected code and requirements, failure modes, dependency assumptions and mapping gaps.
- Evidence plan: the observation each claim requires, inputs, expected outcomes and why these checks address the agreed scope.
- Execution record: exact revisions or source hashes, commands, environment, actual results and rerun instructions.
- Decision record: findings, unresolved questions, exclusions, exceptions and the person or policy authorized to decide.
- Reusable checks: regression evidence and the conditions that would make it stale on the next change.
What this example actually establishes
Both versions return 403 for the other tenant. The candidate nevertheless changes the stored label. The authorized-update control also runs. Across two cases and two implementations, the fixture records four calls. That is evidence of one concrete regression, not general migration readiness.
Inspect the observations and download the source and captured results.
Choose who takes the fix forward
Audit alongside development: your team implements the fix. Proof supplies the requirement, reproducer, wider checks to consider and the evidence needed to assess closure. Ongoing scope keeps these records and checks current as your team changes the component.
Software factory: include implementation and renewed verification in the engagement. The handoff adds the proposed patch and its Change record. A reproduced defect is not reported as fixed until the fixing revision has supporting evidence.
Agree access, approval authority, price and timing after inspecting the build and verification needs. Acceptance, merge and deployment remain separate decisions.
Retain what the next change can reuse.
Proof connects the recovered or approved requirement, hazard, code, reproducer and fixing Change. That retained context is useful after the audit ends, provided its evidence is checked for freshness.
Judge the engagement by decisions it enables and repeated work it removes. Finding counts alone reward noise. Ask to inspect a public record before choosing the scope of a pilot.
Try it on your software
Audit one component with a clear business consequence.
Bring its build path, one unresolved issue and the person who knows the intended behavior.
Start with the scope above. Keep implementation with your team, or include it through the factory. Both routes leave requirements, evidence and open decisions your engineers can inspect.