Skip to content

Preserve payload group atomicity across block materialization - #67

Merged
andinux merged 2 commits into
mainfrom
codex/block-payload-group-atomicity
Sep 22, 2026
Merged

andinux merged 2 commits into
mainfrom
codex/block-payload-group-atomicity

Conversation

@marcobambini

@marcobambini marcobambini commented Sep 19, 2026 •

Copy link
Copy Markdown
Member

A payload group containing ordinary and block columns could be partially committed: a block-table write failure left the ordinary values and metadata applied while the block value was missing or stale.

The pre-block flush now writes pending ordinary columns without releasing the PK-group savepoint or advancing the applied-row count. The boundary stays open through block materialization and closes only at the actual PK/table/source-version boundary. A block failure rolls back the whole group, including ordinary columns, metadata, blocks and resurrection changes. The checkpoint stays unchanged, and replay succeeds once the cause is removed.

Validation

  • 120 forced-failure cases: inserts, updates, resurrection, 3–10 blocks, autocommit and caller-owned transactions.
  • Rejection of first (36 cases), middle (58 cases) and last (26 cases) blocks.
  • Exact before/after comparison of data, metadata and blocks in both directions; unchanged checkpoint and preserved caller transaction.
  • Successful retry and duplicate replay for every case.
  • Core and audit suites pass under ASan/UBSan, including SQLite itself; zero outstanding SQLite memory.
  • Independent PostgreSQL 18.6 compatibility suite: 521 checks pass.
  • Negative control: restoring the old flush fails 360 assertions in the new tests.

docs/internal/block-group-atomicity.md documents the boundary and tests.

This is one of three independent follow-ups to #64, now based on main after #64, #65 and #66 were squash-merged. No deployed cloud server was exercised or modified.

The branch used to carry two more commits, both dropped in the rebase. One was the cloud-integration reliability commit, identical to the one merged with #65. The other only moved the block test to avoid a conflict with #65's test, and the rebase resolves that conflict directly.

Validation after the rebase

make unittest passes (audit regressions and integration bootstrap: 0 failures). Full PostgreSQL suite with ON_ERROR_STOP=on: 553 checks pass on standalone PostgreSQL 17 and 551 on Supabase (test 39's lock-contention part skips there by design). CI run 35751527541 passed on the rebased head a78607c: 37 jobs, only the release job skipped. The first attempt's postgresql17-macos-x86_64 job built fine but timed out uploading its artifact; a re-run passed.

@andinux
andinux changed the base branch from pg-fixes11092026 to main September 22, 2026 16:03
@andinux
andinux force-pushed the codex/block-payload-group-atomicity branch from 5338029 to a78607c Compare September 22, 2026 16:03
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@andinux
andinux merged commit 983853a into main Sep 22, 2026
38 checks passed
@andinux
andinux deleted the codex/block-payload-group-atomicity branch September 22, 2026 17:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants