Topic · Schema and API migration

How do I prove a database or API migration didn't change behavior?

Gist

Flyway records that V42 applied. Pact records that the consumer still signed. Neither is a shall that old readers still work after the drop. Proof fails the merge when a signed migration rule has no witness, with the same command on every push.

proof audit --fail-level warn

Keep Flyway. Keep Pact. Same-input same-output is proof differential fuzz on a declared transform, not the schema file.

Request demo Version vs shall

01 · The version

A green migration is not a kept promise.

The version table moved. Half the fleet still runs the previous binary. That binary reads a column the migration just dropped.

The rewrite that already has a ledger lives on characterization testing and mirrors. The empty start lives on replace a component with no specs. This page is the migration job: schema or API, Flyway or Pact already in the tree, and nobody can point at a check that the sold behavior survived the cutover.

Flyway, Liquibase, Alembic version files. They do that job. They do not compile “old application-version N-1 still reads every column this migration leaves behind.” Pact versions a consumer contract. It does not compile “range scans still return the same projected keys after the engine swap.”

-- V42__drop_settled_at.sql
ALTER TABLE invoices DROP COLUMN settled_at;

Checksum matches. Version table is 42. Tests against the new binary pass, because the new binary never selected that column. Workers still on N-1 raise “column does not exist.” Rollback is already blocked: the data is gone. The catalog name for the missing shall is forward_compatible. List the family with proof catalog list --domain db_migration. proof audit --fail-level warn fails when that shall has no witness on this change.

02 · The exhibit

Same cutover. A version, or a shall.

The version is a row in a table. The shall is this push. Click the tabs.

Flyway / Pact

Looked at checksum, version, signature
Old binary still reads not in scope

V42 applied

The shall

When ApplyMigration deploys schema-version N, it shall not drop a column application-version N-1 still reads. Expand, then contract.

Not in the version table

Flyway / Pact

Still a checksum. Still a signed contract. Still no N-1 reader.

Still the version

Proof

Ask does this change witness forward_compatible
Gate audit --fail-level warn

Merge blocked if unwitnessed

Same cutover. A version, or a shall. Click the tabs.

Who	What they notice	What they lose
Flyway / Liquibase / Alembic	The file applied. The checksum matches. The version table moved.	They version SQL. They do not compile expand-then-contract. Keep them.
Pact / contract tests	Whether the consumer still signed the shape it calls	A signed shape is not range-scan order after an engine swap. Keep Pact.
Golden master / characterization	Observed bytes from the old system	Useful when you can record I/O. That job lives on characterization.
Proof	The migration shall still has a witness, or the merge stays red	Proof does not apply SQL. It does not prove identical rows by itself. Jama still authors the programme.

The three classes on a schema change, from the catalog:

proof catalog list --domain db_migration
proof catalog show forward_compatible
proof catalog show reversible_or_documented
proof catalog show locks_bounded

proof audit --fail-level warn

forward_compatible: deployable while old code still runs. Expand, then contract, never a single PR that drops a column the previous binary reads. reversible_or_documented: a tested down-migration, or a recovery runbook when reverse is impossible. locks_bounded: the lock window is named, not “it was fast on staging.”

If the job is two implementations of one transform, and you need them to agree on concrete inputs, that is proof differential fuzz <transform-key>. Mirror coverage parity is not that. Mirror proves the ledger moved. Differential proves the two shims disagreed on an input, shrinks it, and files an unpromoted known issue. We have not run Flyway against that gate on a frozen corpus of invoice schema. The loss is named, not scored.

03 · The honest loss

A red Proof audit is not a migrated database.

If nobody signed “old readers still work,” there is nothing for proof audit --fail-level warn to fail on except the suite you already have.

Proof does not apply migrations. Flyway still does. Proof does not own consumer-driven contracts. Pact still does. Proof does not prove identical input-output by installing the catalog. That is proof differential fuzz on a declared transform, with two complete shims. A transform with fewer than two shims is skipped as information.

Mirror coverage parity is not behavioral equivalence. Both sides can carry a negative cell and still reject different inputs. Never describe a completed mirror as “the new API matches the old one.” Jama still wins at programme authoring. 100% MC/DC on the written decision still misses a partition that was never a condition.

The rewrite H1 lives on characterization testing and mirrors. The empty-start H1 lives on replace a component with no specs. The money-path H1 lives on fintech transaction logic.

Request demo How the install is bought

04 · Nearby questions

What people type next.

How do I rewrite a legacy system without introducing regressions? A rewrite with a ledger. Characterization testing and mirrors.

How do I safely replace a legacy component that has no specs and no tests? There is nothing honest to characterize. Empty-start replacement.

How do I verify a rewrite behaves identically to the old system? Identical output is proof differential fuzz on a declared transform, not mirror_complete. FAQ on the characterization URL. Do not mint a twin.

What's the safest way to modernize legacy software that nobody fully understands? The unread tree you cannot throw away yet. Unknown-system modernize.