Check migration behavior through outputs and durable effects

Compare the behavior the migration must preserve

For a database or API migration, define the observations that matter before comparing implementations: response fields, error semantics, stored state, ordering and supported consumers. Agreement between old and new versions is useful only within that declared scope. Both versions can agree on a defect, so include expectations justified by the requirement.

The runnable example below simulates an API implementation cutover. It makes real local calls against separate SQLite databases and rereads committed state through fresh connections. It does not run a production schema migration, a network service or live traffic.

The response agrees while the database changes

The illustrative contract, EX-TENANT-1, allows a tenant to update its own document. A request from another tenant must receive a rejection and leave stored rows unchanged.

The candidate handler deliberately commits the label update before checking ownership. Both handlers return a rejection, but only the old implementation preserves the original row. The runner also executes an authorized update as a positive control. Across two cases and two implementations, it records four actual API calls.

Old:       authorize → update → commit → respond
Candidate: update → commit → authorize → reject

Unauthorized case, stored row after the call:
old:       [1, "alpha", "quarterly"]
candidate: [1, "alpha", "revised"]

These are observed teaching-fixture outputs, not a customer incident. The candidate fails one of the two requirement cases. The old implementation passes both tested cases; that does not establish its correctness for untested inputs.

Record who authorized the comparison rule

The fixture permits independently generated correlation IDs, provided each response contains a nonempty string. The runner checks that condition before ignoring the exact value. The rule does not permit dropping status, error semantics or stored state.

The disposition record explicitly attributes that rule to a fictional service owner in an illustrative scenario. It rejects the candidate's unauthorized write and lists repair and follow-up work. This is a teaching decision record, not a real customer's approval or release authorization.

In a real migration, obtain the corresponding decision from the authorized owner. Record the field or effect being normalized, the reason, scope and unresolved risks. A comparator that silently removes every difference can turn a behavioral regression into a passing report.

Run the old and candidate implementations

Download the source and captured results, extract the archive, then choose a new output directory:

cd evidence-methods
python3 run.py --output rerun-results

The runner uses Python 3.9 or later with its sqlite3 module. It needs no pip packages or network access and refuses to overwrite retained results. Exit zero means the teaching outcomes match the expected contrast, including the deliberately rejected migration candidate.

Captured October 3, 2026 at 09:30:30 UTC with Python 3.14.7 and SQLite 3.53.4. The result includes source/input hashes, command, runtime and individual observations. Result SHA-256: 8819dd948702332953a3cf281fe8b70ec7f8a4fa81f2509cda04a9a166596f92. Hashes identify these bytes; they are not an independent signature or proof of origin.

Add deployment evidence beyond this fixture

A schema cutover adds compatibility questions this API example does not exercise. If a new binary no longer reads a column, an older worker may still require it. Inventory the versions that will coexist and test their actual readers and writers against the proposed schema.

1. Expand: add the replacement while retaining supported old fields. Exercise old and new consumers.
2. Migrate: backfill and reconcile values, including partial failures and ordering.
3. Switch: move readers after the required evidence is available and record the deployed-version inventory.
4. Contract: remove retired fields only after their supported consumers are gone and recovery has been assessed.

Keep the migration runner, contract tests and operational controls. Proof can connect their evidence to requirements and known issues. Migration catalog obligations such as forward_compatible and locks_bounded help identify work to consider; selecting an obligation does not execute or satisfy it.

Choose further checks for concurrency, rollback, lock duration, data volume and every supported consumer. The four executions here establish one concrete regression under a small contract. They do not establish general behavioral equivalence or migration readiness.

Source and method boundaries

Python's official sqlite3 documentation, checked October 3, 2026, documents explicit commits and reopening a connection to read stored data. The example uses that mechanism and does not generalize SQLite's transaction behavior to another database. The source, inputs, observed rows and illustrative disposition are all included in the downloadable bundle.