GitHub required checks
- Looked at go test ./... exit 0
- refund:idempotent not a required input
Topic · A required check on the shall
Gist
Add Proof as a required check. proof audit --fail-level warn fails the merge when an approved shall has no witness. The model can still open the PR. GitHub will not merge it.
proof audit --fail-level warn
Keep GitHub for branch protection. Keep CodeRabbit for the comment thread. Keep SonarQube for new-code rules. None of those read the shall.
01 · The missing required check
Branch protection already blocks a red suite. It does not ask whether refund:idempotent still has a witness.
The wrap around the write lives on guardrails for autonomous coding agents. Catching a silent break after the fact lives on when an agent silently breaks a requirement. The CodeRabbit compare keeps the comment bot. This page is the step on the merge: a required check whose input is the approved shall, not the diff.
An agent opens a refund PR. go test ./... is required and green. CodeRabbit left four comments. SonarQube's new-code gate is green because the model also wrote the test. The second POST of the same refund id still moves money twice. Nothing in the required list read
When the same refund id is posted twice, the ledger shall move money once.
# required checks today
tests: go test ./... # green
review bot: comments on the diff # not a status
sonar: new-code rules # green
# SYS-REQ-40:refund:nominal // present
# SYS-REQ-40:refund:idempotent // absent
# merge: allowed
obligation_evidence_complete is a verify-stage warning by default. Wire
proof audit --fail-level warn
as a required status and the missing idempotent fails the PR the same way a red suite does. GitHub still hosts the merge. The shall is now an input.
02 · The exhibit
The suite is green. The second refund still posts. Click the tabs.
GitHub required checks
The shall
When the same refund id is posted twice, the ledger shall move money once.
No idempotent annotationThe agent PR
Still a green suite on the inputs the model picked. Still comments on the diff. Still a new-code score.
Tests keptProof
Required tests, or the required audit. Click the tabs.
| Step | What it buys | What it loses |
|---|---|---|
| GitHub required checks / branch protection | Hosts the merge. Blocks a red job you named. | Does not know the shall. Keep it. Proof is a job on it, not a replacement. branch protection is GitHub's product SERP, not this H1. |
| CodeRabbit / review bots | Comments on the agent diff before you look. | A comment is not a required status. Keep the bot. Compare: Proof vs CodeRabbit. |
| SonarQube quality gate | New-code rules and coverage on this PR. | Smells, not obligations. The quality-gate H1 lives on Proof vs SonarQube. |
| OPA / Gatekeeper | Admission control for cluster objects. | Kubernetes policy, not a pull-request shall. Wrong surface. Do not chase that SERP from this URL. |
| Proof | Each required evidence type on the obligation still has an annotated test, or the required check stays red. | Proof does not invent the shall. Proof is not GitHub. Jama still authors. |
The check reads the obligation checklist, then the triple on the test. Mark the job required in the same place you already require tests.
# .github/workflows/proof.yml
- name: requirement gate
run: proof audit --fail-level warn
proof audit --check obligation_evidence_complete --verbose
proof catalog show refund
// SYS-REQ-40:refund:nominal
// SYS-REQ-40:refund:idempotent
proof audit --fail-level warn
plain class: nominal only.
Classes that name a second evidence type, including idempotent refund, need that type too.
A comment without a run is not a witness. A required suite without that annotation is still a miss.
We have not run this gate against a frozen corpus of agent PRs next to branch protection alone. The loss is named, not scored.
03 · The honest loss
If nobody wrote the refund rule, there is nothing for obligation_evidence_complete to fail on except the checklist you already have.
Proof does not host GitHub. Branch protection, the merge queue, and required status checks stay GitHub's.
Proof does not sandbox the model. Cursor rules and CLAUDE.md still constrain the write. That wrap lives on
the guardrails page.
A warning-by-default check is silent at the default fail level. The gate is
proof audit --fail-level warn.
A transform with no obligation checklist is skipped as information. Jama still authors the programme.
CodeRabbit still comments. SonarQube still scores new code.
The wrap H1 lives on guardrails. The silent-break H1 lives on silently breaks a requirement. The speed H1 lives on ship faster without more bugs. The incumbent H1 lives on Proof vs CodeRabbit. Do not mint a twin of any of those.
04 · Nearby questions
What's a reliable way to gate AI pull requests on correctness? Same required check. Same command. Not a second URL.
How do I catch when an AI agent silently breaks an existing requirement? Reverse suspect after the fact, not the merge step. Silently breaks.
What guardrails should I put around autonomous coding agents? The wrap around the write, not the required status. Guardrails.
How do I let AI agents ship faster without shipping more bugs? Do not throttle the model. Same command, different H1. Ship faster.
Proof vs CodeRabbit? Comments versus a required witness. Proof vs CodeRabbit.