Fix failed SQLite payload commit cleanup; document remaining OOM recovery - #65
Merged
Merged
Conversation
andinux
force-pushed
the
codex/sqlite-apply-transaction-cleanup
branch
from
September 21, 2026 18:15
a35c0eb to
063d323
Compare
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
PostgreSQL apply never owns the transaction, so the SQLite failed-commit cleanup does not apply there. Cover that: a deferred foreign key fails the caller's COMMIT with 23503 and rolls back rows, metadata, checkpoint and caller work together; autocommit keeps nothing; a write failing inside a caller savepoint leaves the caller's transaction usable for redelivery. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… tenant history The rowset and paths tests ignored the receive JSON, so an apply error reported in receive.error looked the same as a slow server. Fail on receive.error, and on timeout report rows present/received, elapsed time and the last receive.lastFailure. A fresh receiver first downloads the whole shared tenant history (~113k rows); the server took ~25s to prepare it, past the ~22s budget, on main too. Raise both polls to 120 attempts; they still stop as soon as the rows arrive. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
andinux
marked this pull request as ready for review
September 21, 2026 22:39
andinux
added a commit
that referenced
this pull request
Sep 22, 2026
Test 62 is now 62_deferred_fk_caller_commit.sql, merged with #65. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Sep 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Payload apply can return a deferred foreign-key or busy-commit error while leaving its internally opened SQLite transaction active. Uncommitted rows remain visible and a later
BEGINfails.This change centralizes error cleanup, rolls back a failed transaction only when apply started it from autocommit mode, preserves the original error and committed prefixes, and excludes rolled-back rows from applied statistics. It also rejects a failed group-savepoint open and fixes a primary-key-name allocation leak found by fault injection. Caller-owned transactions remain under the caller's control.
Validation
database_in_transaction()is always true under SPI), so the newROLLBACKis unreachable.test/postgresql/62_deferred_fk_caller_commit.sqlpins this down: a deferred foreign key lets the apply succeed and fails the caller'sCOMMITwith23503, rolling back rows, metadata, checkpoint and caller work together; autocommit keeps nothing; a write failing inside a caller savepoint (P0001) leaves the caller's transaction usable, and redelivery applies. Full suite withON_ERROR_STOP=on: 528 checks pass on standalone PostgreSQL 17, 526 on Supabase (test 39's lock-contention part skips there by design).Remaining limitation: allocation failures (#74)
The engine-level OOM case from the report is not fully fixed and is tracked in #74. When an allocation fails mid-apply, SQLite refuses the cleanup
ROLLBACKfrom inside the running SQL function, so 126 of the 348 swept allocation points (0–347, ordinary build) still leave the apply's transaction open. There are zero leaks, and every retry succeeds once the host rolls back. The diagnostictest/stress/payload_oom.cdeliberately exits nonzero for those cases and is not part ofmake unittest.docs/internal/apply-transaction-cleanup.mddocuments the fix, reproduction command, results and remaining recovery requirement.This is one of three independent follow-ups to #64, now based on
main(rebased after #64 was squash-merged as49ba88e). Apart from running the e2e suite against the shared CI test tenants, no live cloud endpoint was used or modified.CHANGELOG.mdrecords the fix under[Unreleased]: it will ship in a later release together with the other open follow-ups, so there is no version bump here.Cloud integration reliability
The shared chunked-tenant job is serialized across branches with a queued concurrency group. This prevents a different PR from writing during the negative-cache test's required idle phase; all idle assertions remain intact, and other platforms still run in parallel.
Fresh receivers now wait for actual received rows and expected fixture data rather than requiring the first poll to contain rows. Polling is bounded and SQL/protocol failures abort immediately. A materialized result ensures one network call per attempt. The offline
integration_bootstraptest runs as part ofmake unittest, covering 312 scenarios including delayed/partial delivery, timeout, missing data, protocol errors and malformed responses. Local ordinary and ASan/UBSan runs pass with zero outstanding SQLite memory.The chunked rowset and paths e2e tests now fail on
receive.errorinstead of polling until the timeout, and a timeout reports rows present/received, elapsed time and the lastreceive.lastFailure. With that, the intermittentchunked rowset e2e batch ... was not receivedfailure was traced (reproduced locally on this branch and onmain): a fresh receiver first downloads the whole shared tenant history (~113k rows), which the server took ~25 s to prepare, past the old ~22 s budget. Both polls now allow 120 attempts and still stop as soon as the rows arrive; the full e2e suite passes locally.Latest CI verification
Workflow run 35661808765 completed successfully on
1d22099: 37 jobs passed, with only the release job skipped. All chunked e2e tests passed on Linux x86_64, including the rowset test with the new polling budget.