Topic · Silent requirement break

How do I catch when an AI agent silently breaks an existing requirement?

The PR can look like a refactor. The shall did not move. Proof marks the link suspect when the code is newer than the requirement, then the audit fails the merge.

proof audit --check suspect_clean

Code review still reads the diff. It does not notice that SYS-REQ-012 is now older than the file that claimed to implement it.

Request demo Forward vs reverse

01 · The silent break

New-code review and a matrix are different jobs.

Verifying a function the model just wrote is one question. Catching that it quietly dropped a shall you already approved is another.

The first job is verify AI agent code: is this new function actually correct. The matrix job is the traceability page: can I re-derive which code satisfies which shall. This page is the delta after a commit. The agent changed a file that was already linked. Nobody touched the requirement. The tests the model also wrote still pass.

A link is suspect when either side moved since the link was last verified. Forward: the requirement is newer than the artifact. Reverse: the code is newer than the requirement. The silent agent break is reverse. That is the case proof audit --check suspect_clean is built to fail.

02 · The exhibit

Two directions. Only reverse is the agent sneaking past the shall.

Same graph. Two timestamps. The review comment on the diff does not see either of them.

Diff review

Sees lines in this PR
Green tests the agent wrote

Looks like a refactor

SYS-REQ-012

The file is newer than the shall. The review comment does not see the timestamp.

Not consulted

The PR

Still a clean thread.

Looks like a refactor

Proof

Check suspect_clean
Fail code newer than the requirement

Merge blocked

Same commit. Two clocks. Click the tabs.

Job	Diff / PR bot	Proof
What it sees	Lines in this pull request.	Git timestamps on both ends of the link.
Forward	The shall moved. The file might look untouched.	`SYS-REQ-005 --> implemented_by` is suspect because the requirement is newer.
Reverse	A refactor. Tests green. Comment if the style bothers you.	`SYS-REQ-012 <-- implemented_by` is suspect because the code is newer.
Stops the merge?	If someone reads the thread and blocks.	Yes, when `suspect_clean` is on the audit.
Who can clear it	The same agent can resolve its own review comments.	A human `proof trace review`, or a citation the repo can re-check. Agent actors are refused on a rationale review.

We have not run a frozen corpus of agent-broken shalls against proof audit --check suspect_clean. The loss is named, not scored. CodeRabbit and Copilot review still read the diff. They stay. They are not a substitute for the timestamp on the link.

proof trace suspect
proof audit --check suspect_clean
# >>> [VERIFICATION] suspect_clean -- 0 suspect links

proof trace review SYS-REQ-012 --citation pkg/trace/linker.go

Freshness is the latest Git commit on a clean tracked file, not checkout mtime. Dirty or untracked files still use filesystem mtime, so a local edit invalidates the review immediately. proof trace suspect --all is whole-project inspection. The audit check stays on the verification scope.

03 · Honest loss

Proof does not sandbox the agent.

It does not stop the model writing the diff. It fails the merge after the file is already newer than the shall.

A rationale review is a human signature. The command refuses agent actors for that path because the only thing standing behind free text is who signed it. A --citation is different: the path has to be one of that requirement's own trace targets, and the repo can re-check the claim. Neither of those is a runtime monitor, a secret scanner, or a prompt rule in Cursor.

Jama still authors the programme. Proof does not replace an ALM. It does not prove the new function is correct. That is the verify-AI page. It does not rebuild the matrix from a spreadsheet. That is the RTM page. After week four the gate is in your CI. The engagement can stop. The bar does not. That install is the audit.

04 · Next

What people type next.

How do I verify code that an AI agent wrote is actually correct? New function, new test, same context window. Verify AI agent code.

How do I keep my spec and my code from drifting apart over time? That is the matrix, not the delta. Requirements traceability matrix.

What's a reliable way to gate AI pull requests on correctness? The compare page against the PR bot. Proof vs CodeRabbit.

What guardrails should I put around autonomous coding agents? Merge exit, not a sandbox. Guardrails around autonomous coding agents.