Skip to content

Cooling-stage clock sweep (batched sweep + HOT/COOL evictor + strategy removal) - #34

Open
gburd wants to merge 5 commits into
masterfrom
bcs
Open

gburd wants to merge 5 commits into
masterfrom
bcs

Conversation

@gburd

@gburd gburd commented Jul 9, 2026

Copy link
Copy Markdown
Owner

Cooling-stage clock sweep for the buffer manager — CI check.

This PR is against gburd/postgres:master (my fork), not upstream, purely
to exercise CI on the three-patch series before posting to pgsql-hackers.

  1. Batch the clock sweep to reduce nextVictimBuffer atomic contention
  2. Replace the usage_count clock sweep with a cooling-stage evictor
  3. Remove BufferAccessStrategy; scan resistance is now intrinsic

Net +510/-1642 over 83 files (almost all deletion is patch 3).

Each commit builds -Werror + cassert + injection_points and passes
regress/regress (245 tests) independently; verified on both the fork base
and a clean upstream/master rebase. See the draft cover letter for the
design reasoning and the m6i / r8i (6-node NUMA) / huge-pages / local-NVMe
real-IO benchmark data.

Not for merge — this is the review/CI vehicle for the -hackers submission.

@github-actions

github-actions Bot commented Jul 13, 2026 •

Copy link
Copy Markdown

📜 Change history & discussion (Agora / pg.ddx.io)

I have everything needed.

🧵 Related discussion

  • Direct origin thread: [PATCH] Batched clock sweep to reduce cross-socket atomic contention — Greg Burd, 2026-04-25 (21 messages, runs through late July 2026). This is the PR author's own thread and the design/rationale source for every substantive change in the PR. Confidence: high.
    • Original patch (from Jim Mlodgenski, tweaked by Burd): each backend claims a batch of 64 consecutive buffer IDs from nextVictimBuffer via one pg_atomic_fetch_add_u32 instead of one atomic per tick. Preserves "every buffer visited once per pass," thins the atomic contention. Gated on pg_numa_get_max_node() >= 1, falls back to batch size 1 (bit-for-bit original) on single-socket. This maps to the commit "Bound clock-sweep work without changing the replacement state."
    • Andres Freund's core objection (reply, 2026-04-27): batching that is independent of contention and of per-backend usage rate is suspect. Low-rate backends sit on a checked-out batch without decrementing usage_count, giving those buffers undue eviction protection. He pushed instead for a partitioned sweep with balancing as a better NUMA building block. The author's "why usage_count doesn't matter" argument assumes uniform replacement rate, which Andres rejects as benchmark-only.
    • Ring-buffer / strategy removal (the PR's "strategy removal" piece, "0003"): Ants Aasma (2026-07-22) and Andres both corrected Burd's rationale. Burd conceded his "rings defer writeback to bgwriter" framing was flat wrong: BAS_BULKWRITE keeps writes on the originating backend, and StrategyRejectBuffer only fires for BAS_BULKREAD. Ants' standing concern: removing rings needs a replacement backpressure mechanism for bulk writers. Not resolved in-thread.
    • Evaluation methodology (Andres, 2026-07-22): hit ratio (e.g. pgbench -S) is a poor metric for replacement quality — which buffers are evicted matters (inner index pages cause fan-out stalls). Relevant to any HOT/COOL evictor claims.

🔗 Related commits / prior art

📋 Commitfest

  • No commitfest entry found by author email (greg@burd.me) or by thread Message-ID.

🧭 Context for reviewers

  • The PR bundles work well beyond the mailing-list patch. The thread proposed only the batched sweep (the one commit fecdc2b911a); the PR title adds HOT/COOL evictor and strategy (ring buffer) removal, which in the thread are unmerged, actively-contested directions, not accepted designs.
  • The scope-signal is noise: this PR mixes the actual buffer change with unrelated edits to arrayfuncs.c, tsvector.c, xlog.c, logical replication (logicalctl.c, slotsync.c), a deleted recovery test (051_effective_wal_level.pl), plus a fork's dev tooling (.clangd, flake.nix, shell.nix, pg-aliases.sh, a LaTeX paper, nix patches). These are not part of the clock-sweep discussion and look like a dirty branch / rebase artifacts. Scrutinize the test deletion and the array/tsvector/xlog hunks separately.
  • Two unresolved upstream blockers to carry into review: (1) Andres' argument that fixed, rate-blind batching distorts usage_count under non-uniform backend load, with partitioned-sweep offered as the preferred alternative; (2) Ants' point that removing bulk-write rings removes backpressure on bulk writers with no replacement proposed. The author already retracted his writeback-deferral justification for the ring removal.
  • Provenance disclosed by the author: the initial exploration used an AI assistant (Kiro). Benchmarks in the thread are Jim Mlodgenski's; the author had not independently reproduced them as of the opening message.

Generated by pg-history via the Agora MCP server (pg.ddx.io).

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 OCR found 12 issue(s).

  • 12 inline, 0 in summary
  • ⚠️ 26 warning(s) during review

Comment thread contrib/amcheck/sql/check_heap.sql Outdated
Comment on lines +45 to +46
SELECT sum(reads) AS stats_bulkreads_before
FROM pg_stat_io WHERE context = 'bulkread' \gset
FROM pg_stat_io WHERE context = 'normal' AND object = 'relation' \gset

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The \gset variables are still named stats_bulkreads_before/stats_bulkreads_after, but this query now deliberately measures the normal context (the comment above was rewritten to say the reads are no longer bulkreads). The names now contradict what they hold. Since these exact lines are already being touched, rename them to reflect the new semantics (e.g. stats_normal_reads_before) for both \gset sites and the comparison on the following line. This also requires the matching update in contrib/amcheck/expected/check_heap.out.

Suggested change
SELECT sum(reads) AS stats_bulkreads_before
FROM pg_stat_io WHERE context = 'bulkread' \gset
FROM pg_stat_io WHERE context = 'normal' AND object = 'relation' \gset
SELECT sum(reads) AS stats_normal_reads_before
FROM pg_stat_io WHERE context = 'normal' AND object = 'relation' \gset

forknum = BufTagGetForkNum(&bufHdr->tag);
blocknum = bufHdr->tag.blockNum;
usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
usagecount = BUF_STATE_GET_COOLSTATE(buf_state);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This silently changes the user-visible semantics of the pg_buffercache.usagecount column. BUF_STATE_GET_USAGECOUNT returned the full clock-sweep count (0..BM_MAX_USAGE_COUNT, historically 0..5), whereas BUF_STATE_GET_COOLSTATE returns only bit 0 (0=COOL, 1=HOT). The column is still declared smallint and documented as "Clock-sweep access count", so users querying usagecount will now silently get only 0/1 with no doc or column-name update. This is a backward-incompatible behavior change that needs the documentation (pgbuffercache.sgml) updated and, arguably, the column renamed to reflect HOT/COOL semantics. (high confidence)

{
buffers_used++;
usagecount_total += BUF_STATE_GET_USAGECOUNT(buf_state);
usagecount_total += BUF_STATE_GET_COOLSTATE(buf_state);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

usagecount_total now accumulates only 0/1 per buffer instead of 0..BM_MAX_USAGE_COUNT, so the derived usagecount_avg column changes range/meaning without any doc update. The SGML still labels it a clock-sweep average. Update the documentation to match the new HOT/COOL semantics. (high confidence)

CHECK_FOR_INTERRUPTS();

usage_count = BUF_STATE_GET_USAGECOUNT(buf_state);
usage_count = BUF_STATE_GET_COOLSTATE(buf_state);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pg_buffercache_usage_counts() now buckets buffers only into indices 0 and 1 (COOL/HOT); the usage_count output column will never exceed 1. This is fine for array bounds since BM_MAX_USAGE_COUNT is redefined to BUF_COOLSTATE_HOT (=1), but the SQL column name usage_count and its doc ("A possible buffer usage count") are now misleading. Update pgbuffercache.sgml and the expected regression output to reflect the reduced value domain. (high confidence)

Comment thread src/backend/access/heap/heapam.c Outdated
Comment on lines +398 to +401
allow_sync = (scan->rs_base.rs_flags & SO_ALLOW_SYNC) != 0;
}
else
allow_strat = allow_sync = false;

if (allow_strat)
{
/* During a rescan, keep the previous strategy object. */
if (scan->rs_strategy == NULL)
scan->rs_strategy = GetAccessStrategy(BAS_BULKREAD);
}
else
{
if (scan->rs_strategy != NULL)
FreeAccessStrategy(scan->rs_strategy);
scan->rs_strategy = NULL;
}
allow_sync = false;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The if body is now a single statement after removing the allow_strat assignment, but the braces are retained. PostgreSQL style (and pgindent-adjacent convention) omits braces around single-statement bodies. Since this block was directly modified by the diff, drop the braces to keep the code reading as if it had always been written this way:

	if (!RelationUsesLocalBuffers(scan->rs_base.rs_rd) &&
		scan->rs_nblocks > NBuffers / 4)
		allow_sync = (scan->rs_base.rs_flags & SO_ALLOW_SYNC) != 0;
	else
		allow_sync = false;

Confidence: moderate.

Suggested change
allow_sync = (scan->rs_base.rs_flags & SO_ALLOW_SYNC) != 0;
}
else
allow_strat = allow_sync = false;
if (allow_strat)
{
/* During a rescan, keep the previous strategy object. */
if (scan->rs_strategy == NULL)
scan->rs_strategy = GetAccessStrategy(BAS_BULKREAD);
}
else
{
if (scan->rs_strategy != NULL)
FreeAccessStrategy(scan->rs_strategy);
scan->rs_strategy = NULL;
}
allow_sync = false;
allow_sync = (scan->rs_base.rs_flags & SO_ALLOW_SYNC) != 0;
else
allow_sync = false;

else
elevel = DEBUG2;

/* Set up static variables */

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is now dangling. It solely described the removed vac_strategy = bstrategy; assignment; the only other static in this file (anl_context) is set up in do_analyze_rel, not here. Leaving an empty comment block referring to nonexistent code violates the minimal-diff/comment-accuracy discipline. Remove the comment (and the now-doubled blank line).

Comment thread src/backend/storage/aio/read_stream.c Outdated
Comment on lines 1365 to 1368
read_stream_next_block(ReadStream *stream)
{
*strategy = stream->ios[0].op.strategy;
return read_stream_get_block(stream, NULL);
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With strategy removed, read_stream_next_block() no longer has any distinguishing behavior: it is now an exported thin wrapper around read_stream_get_block(stream, NULL) and has no callers anywhere in the tree (verified: only the callback functions apw_read_stream_next_block and collect_corrupt_items_read_stream_next_block match by name; no invocation of read_stream_next_block(...) exists). Its sole prior reason to exist over calling the internal helper directly was reporting the strategy, which this patch deletes. Recommend removing the function (and its prototype in read_stream.h) as dead transitional scaffolding, or, if it is intentionally retained as public API, keep it clearly justified. Leaving a caller-less exported wrapper contradicts the minimal-diff / YAGNI discipline. (moderate confidence)

Comment thread src/backend/storage/buffer/freelist.c Outdated
Comment on lines +136 to +142
current = pg_atomic_read_u32(&StrategyControl->nextVictimBuffer);
if (current >= (uint32) NBuffers)
{
wrapped = current % NBuffers;
if (pg_atomic_compare_exchange_u32(&StrategyControl->nextVictimBuffer,
&current, wrapped))
StrategyControl->completePasses++;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

completePasses can be under-counted under concurrency, corrupting bgwriter pacing. StrategySyncStart() relies on nextVictimBuffer staying below 2*NBuffers so its pending-wrap compensation (nextVictimBuffer / NBuffers) is at most 1. With batching, many concurrent backends can each land a fetch_add(batch_size) before any of them reaches the spinlock, so the shared counter can grow to NBuffers + MaxBackends*batch_size. When it exceeds 2*NBuffers, wrapped = current % NBuffers subtracts multiple NBuffers-multiples but completePasses is incremented only once, permanently losing pass counts. This is reachable in the small-NBuffers case (minimum 16, batch capped at NBuffers): 2*NBuffers is easily exceeded by concurrent claims. Consider incrementing completePasses by current / NBuffers instead of a flat +1, and/or bounding growth so the counter cannot exceed 2*NBuffers.

Suggested change
current = pg_atomic_read_u32(&StrategyControl->nextVictimBuffer);
if (current >= (uint32) NBuffers)
{
wrapped = current % NBuffers;
if (pg_atomic_compare_exchange_u32(&StrategyControl->nextVictimBuffer,
&current, wrapped))
StrategyControl->completePasses++;
current = pg_atomic_read_u32(&StrategyControl->nextVictimBuffer);
if (current >= (uint32) NBuffers)
{
wrapped = current % NBuffers;
if (pg_atomic_compare_exchange_u32(&StrategyControl->nextVictimBuffer,
&current, wrapped))
StrategyControl->completePasses += current / NBuffers;

Comment thread src/backend/storage/buffer/freelist.c Outdated
Comment on lines +148 to +149
MyBatchPos = start;
MyBatchEnd = start + batch_size;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A batch can straddle the NBuffers boundary without any wrap being registered on the shared counter. When start < NBuffers but start + batch_size > NBuffers, the wrap branch (start >= NBuffers) is skipped, yet victim = MyBatchPos % NBuffers wraps internally as MyBatchPos crosses NBuffers. The completePasses increment for that crossing is deferred until a later fetch-add happens to return start >= NBuffers. Between those events, StrategySyncStart() sees nextVictimBuffer already advanced past NBuffers and relies solely on its nextVictimBuffer / NBuffers term to compensate. That compensation is only correct while the counter stays below 2*NBuffers (see the related finding). This coupling is fragile and should be documented or handled explicitly, since a mid-batch wrap is now the common case (batch_size defaults to 32).

Comment thread src/backend/storage/buffer/freelist.c Outdated
Comment on lines +24 to +25
#include "storage/subsystems.h"
#include "port/pg_numa.h"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Include is out of group/alphabetical order: port/pg_numa.h is placed after the storage/* block. Per the tree's convention (and pgindent grouping), it belongs with the other port/* headers, immediately after port/atomics.h.

Suggested change
#include "storage/subsystems.h"
#include "port/pg_numa.h"
#include "storage/subsystems.h"

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 OCR found 43 issue(s).

  • 25 inline, 18 in summary (inline capped at 25)

📄 src/backend/access/nbtree/nbtree.c (L1732-L1732)

The strategy argument was removed from this ReadBufferExtended call, but the comment above still claims "we want to use a nondefault buffer access strategy." This comment is now stale and misleading -- no strategy is passed anymore. Update the comment to reflect that the buffer access strategy is now handled elsewhere (or drop that sentence). (moderate confidence)


📄 src/backend/commands/analyze.c (L124-L124)

The /* Set up static variables */ comment is now stale: this diff removed the only statement it introduced (vac_strategy = bstrategy;), leaving the comment describing nothing. Remove the dangling comment so the code reads as if it had always been written this way (minimal-diff hygiene). (high confidence)


📄 src/backend/commands/vacuum.c (L178-L179)

This removal makes VACUUM (BUFFER_USAGE_LIMIT ...) and ANALYZE (BUFFER_USAGE_LIMIT ...) a hard error. The grammar (gram.y utility_option_list) still accepts any option name as a generic DefElem, so with this branch gone the option now falls through to the unrecognized "VACUUM"/"ANALYZE" option ereport below. This is a backward-compatibility break of a documented, user-visible SQL option, and it breaks existing dump/restore scripts and tooling that pass BUFFER_USAGE_LIMIT. If the intent is to fully retire the feature, the corresponding user-facing surfaces (SQL docs in doc/src/sgml/ref/{vacuum,analyze}.sgml and psql tab-completion in tab-complete.in.c, which still advertise BUFFER_USAGE_LIMIT) must be updated in the same change so the option isn't offered while silently erroring. Confidence: high.


📄 src/backend/storage/buffer/freelist.c (L161-L163)

completePasses accounting is inconsistent with StrategySyncStart's compensation term and can double-count / go backwards.

StrategySyncStart() still returns *complete_passes = completePasses + nextVictimBuffer / NBuffers. But now:

  1. This else if increments completePasses for a batch that crosses the wrap point without wrapping the shared counter. The counter is then >= NBuffers, so a subsequent StrategySyncStart adds nextVictimBuffer / NBuffers (>=1) on top of the increment already applied here -> the same pass is counted twice.

  2. The if (start >= NBuffers) branch above wraps current % NBuffers but increments completePasses by only 1 even when current drifted to e.g. 2*NBuffers; the sync-start compensation term drops from 2 to 0 across that wrap, so the visible pass count decreases.

Because BgBufferSync does passes_delta = strategy_passes - prev_strategy_passes; strategy_delta += passes_delta*NBuffers; Assert(strategy_delta >= 0), a non-monotonic completePasses can trip that assertion and corrupts bgwriter pacing. Confidence: high. The batched wrap accounting needs to be made consistent with StrategySyncStart's nextVictimBuffer / NBuffers term (i.e. don't both pre-count the crossing here and let sync-start count it again).


📄 src/backend/storage/buffer/freelist.c (L137-L142)

This if (start >= NBuffers) wrap path can lose passes when the counter has drifted more than one NBuffers period. current may be >= 2*NBuffers under batching/contention, but wrapped = current % NBuffers collapses it and completePasses is bumped by only 1, while StrategySyncStart's nextVictimBuffer / NBuffers compensation was >1 before the wrap and becomes 0 after. Net effect: the reported completed-pass count can decrease across the wrap, which BgBufferSync's Assert(strategy_delta >= 0) does not tolerate. Confidence: high. Consider incrementing completePasses by current / NBuffers (the number of whole periods being discarded) so the (nextVictimBuffer, completePasses) pair stays monotonic.


📄 src/backend/storage/buffer/freelist.c (L324-L330)

Under force_cool, resetting trycounter = NBuffers on the ref-bit-clear transition (not just on the HOT->COOL demote) means a highly contended, all-HOT pool where PinBuffer keeps re-setting BUF_REFBIT (it does so on every pin, bufmgr.c:3302) can keep this backend clearing ref bits and resetting the counter without ever landing on a reclaimable COOL buffer. The intended "second unproductive full pass -> elog(ERROR, no unpinned buffers)" backstop can then be deferred well beyond the ~3 passes the comment claims. Consider only resetting trycounter on the actual HOT->COOL demotion (real forward progress toward a victim), and treating a bare ref-clear as non-progress so the escalation/backstop stays bounded. Confidence: moderate.


📄 src/backend/storage/buffer/bufmgr.c (L86-L86)

Dead flag: BUF_COOLED is defined here, documented in the SyncOneBuffer header, and set via result |= BUF_COOLED, but no caller ever inspects it. SyncOneBuffer's only cool_if_hot=true caller is BgBufferSync, which reacts only to BUF_WRITTEN and BUF_REUSABLE. Per the minimalism/YAGNI discipline, either wire BUF_COOLED into a consumer (e.g. bgwriter pacing/instrumentation) or drop the flag, its result |= assignment, and the doc line. (high confidence)


📄 src/backend/storage/buffer/bufmgr.c (L2320-L2325)

Comment-drift / misleading placement. This "Admit the newly loaded page COOL" comment sits directly above the unrelated BM_PERMANENT conditional, but the actual COOL admission is the removal of BUF_USAGECOUNT_ONE from the set_bits |= BM_TAG_VALID; line above (a freshly admitted buffer now has coolstate 0 == COOL). As written the comment misleads readers into thinking the BM_PERMANENT block performs the cooling admission. Move the comment to sit above the set_bits |= BM_TAG_VALID; line. (moderate confidence)


📄 src/backend/storage/buffer/bufmgr.c (L2964-L2968)

Same comment-placement issue as the other admission site: the "Admit COOL (probation)" comment is above the BM_PERMANENT conditional rather than the set_bits |= BM_TAG_VALID; line where BUF_USAGECOUNT_ONE was previously set. Relocate it above set_bits |= BM_TAG_VALID; so the comment describes the line that actually performs COOL admission. (moderate confidence)


📄 src/backend/utils/activity/pgstat_io.c (L443-L449)

This comment block is now stale/dangling. The if blocks it described (the B_CHECKPOINTER/B_BG_WRITER/autovac restrictions on the removed bulkread/bulkwrite/vacuum IOContexts) were deleted, leaving the comment sitting directly above return true; describing nothing. Per PG comment discipline (comments must describe what the code does now) and minimal-diff hygiene, remove the orphaned comment so the function reads as if it had always been written this way.

💡 Suggested change

Before:

	/*
	 * Some BackendTypes do not currently perform any IO in certain
	 * IOContexts, and, while it may not be inherently incorrect for them to
	 * do so, excluding those rows from the view makes the view easier to use.
	 */
 
 	return true;

After:

	return true;

📄 src/bin/scripts/vacuumdb.c (L355-L355)

This removes the user-visible --buffer-usage-limit option (getopt entry, handler, validation, and help text), but the corresponding documentation is not removed: doc/src/sgml/ref/vacuumdb.sgml still contains a full <varlistentry> for --buffer-usage-limit (around line 135). A user-visible removal must delete the matching doc entry in the same patch, otherwise the docs describe an option the binary no longer accepts. Please remove the SGML block as part of this change. (high confidence)


📄 src/bin/scripts/vacuuming.c (L267-L272)

This removes the --buffer-usage-limit handling from vacuumdb, a user-visible option shipped since PG16. Together with the coordinated removals in vacuumdb.c and vacuuming.h, this deletes an established CLI feature. This is a backward-compatibility break: existing scripts invoking vacuumdb --buffer-usage-limit=... will now fail with an unknown-option error. Such a removal needs extraordinary justification and a design discussion on -hackers; nothing in the diff explains why. Additionally, doc/src/sgml/ref/vacuumdb.sgml still documents --buffer-usage-limit and is not part of this change set, so the docs will describe an option that no longer exists. (high confidence)


📄 src/include/storage/bufmgr.h (L357-L360)

Dead section header and whitespace churn. All prototypes that lived under "/* in freelist.c /" (GetAccessStrategy, FreeAccessStrategy) were removed, so this comment now labels nothing, and the removal left a stray double blank line before "/* inline functions /". This will trip git diff --check / pgindent style expectations. Remove the empty "/ in freelist.c */" section entirely and collapse to a single blank line.

💡 Suggested change

Before:

/* in freelist.c */


/* inline functions */

After:

/* inline functions */

📄 src/include/storage/bufmgr.h (L221-L222)

API/ABI break on widely-used exported entry points (moderate confidence on impact). ReadBufferExtended(), ReadBufferWithoutRelcache(), and the ExtendBufferedRel*() family drop their BufferAccessStrategy parameter, while GetAccessStrategy()/GetAccessStrategyWithSize()/GetAccessStrategyBufferCount()/GetAccessStrategyPinLimit()/FreeAccessStrategy() are removed outright. These are core buffer-manager APIs used pervasively by out-of-tree extensions; every such extension will fail to compile against this header. On HEAD a signature change is allowed, but wholesale removal of the strategy concept with no replacement is a substantial, non-obvious semantic change (ring buffers gone; scan/VACUUM no longer bounded to a small buffer ring). For pgsql-hackers this needs (a) a reference to the design thread / commitfest entry justifying dropping ring buffers, and (b) confirmation it is a single logical change -- as submitted it bundles at least three independent things (remove BufferAccessStrategy, replace usage_count clock sweep with HOT/COOL cooling state, add NUMA-batched clock sweep), which should be split into separately-committable, individually-bisectable patches.


📄 src/include/pgstat.h (L294-L294)

This change reduces IOCONTEXT_NUM_TYPES from 6 to 2, which shrinks the fixed-stats structs PgStat_BktypeIO/PgStat_PendingIO/PgStat_IO (dimensioned [IOOBJECT_NUM_TYPES][IOCONTEXT_NUM_TYPES][IOOP_NUM_TYPES], lines 328-337). PgStat_IO is a fixed stats kind that is serialized to the persisted stats file. The comment above PGSTAT_FILE_FORMAT_ID (line 216-217) explicitly requires bumping the format ID "whenever any of these data structures change", and on startup pgstat_read_statsfile() only discards a stale file when format_id != PGSTAT_FILE_FORMAT_ID. Since PGSTAT_FILE_FORMAT_ID (0x01A5BCBC) is unchanged in this patch, a stats file written by a pre-patch server will be accepted and read with the new, smaller layout, corrupting the fixed-stats block (and desynchronizing subsequent parsing). Bump PGSTAT_FILE_FORMAT_ID. Unlike catversion.h this stamp is the author's responsibility. Confidence: high.


📄 src/include/storage/buf_internals.h (L136-L137)

BUF_STATE_GET_USAGECOUNT now has no callers anywhere in the tree -- every consumer was switched to BUF_STATE_GET_COOLSTATE. Leaving the old accessor behind is dead code and a footgun: the field it reads now mixes the cooling bit (bit 0) and the ref bit (bit 1), so any new caller that reaches for the familiar name gets a 0..3 value conflating two orthogonal flags. Remove it. (high confidence)


📄 src/include/storage/buf_internals.h (L139-L140)

pgindent-nonconforming multi-line comment: the block opens with text on the same line as /* and closes with an inline */, unlike every other multi-line comment in this header (which puts /* on its own line with asterisk-aligned continuation). This will not survive pgindent cleanly. (moderate confidence)

💡 Suggested change

Before:

/* Cooling state (HOT/COOL) from buffer state -- bit 0 of the field only, so
 * the second-chance ref bit (bit 1) does not perturb the HOT/COOL test. */

After:

/*
 * Cooling state (HOT/COOL) from buffer state -- bit 0 of the field only, so
 * the second-chance ref bit (bit 1) does not perturb the HOT/COOL test.
 */

📄 src/include/storage/buf_internals.h (L191-L191)

BM_MAX_USAGE_COUNT now means "the HOT cool-state value" (=1), not a usage-count maximum, yet it is still consumed as a numeric range bound -- e.g. pg_buffercache sizes usage_counts[BM_MAX_USAGE_COUNT + 1] and iterates over it. Retaining a count-named macro for a boolean cool-state violates POLA and invites a future reader to treat it as a real max count. Prefer a name that reflects the new semantics (e.g. BM_MAX_COOLSTATE) and update call sites, rather than aliasing the old name for the single expression that happens to read naturally. (moderate confidence)

Comment thread .github/ocr/litellm.yaml Outdated
@@ -0,0 +1,41 @@
# LiteLLM proxy config — bridges Open Code Review (OpenAI protocol) to AWS Bedrock.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-ASCII em-dash character (—) in a source/config comment. The project's contribution standards require ASCII-only in source and diffs (no smart quotes, em-dashes, or ellipsis characters). Replace with an ASCII - (or --).

Suggested change
# LiteLLM proxy config — bridges Open Code Review (OpenAI protocol) to AWS Bedrock.
# LiteLLM proxy config - bridges Open Code Review (OpenAI protocol) to AWS Bedrock.

Comment thread .github/ocr/litellm.yaml Outdated
aws_region_name: os.environ/AWS_REGION

# "High effort" review. Claude Opus 4.8 on Bedrock uses *adaptive* thinking
# controlled by output_config.effort. Set it DIRECTLY here — NOT via

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-ASCII em-dash character (—) in this comment. Replace with an ASCII - to comply with the ASCII-only rule for source and diffs.

Suggested change
# controlled by output_config.effort. Set it DIRECTLY here — NOT via
# controlled by output_config.effort. Set it DIRECTLY here - NOT via

Comment thread .github/ocr/pg-history.py Outdated
Comment on lines +175 to +176
import boto3
from botocore.config import Config

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The module docstring promises the script "exits 0 even on soft failures (writes a note)", and every other failure path here honors that (MCP unreachable at line 168, no tools at line 172, Bedrock error at line 212 all write a note to OUT and return). But import boto3 / from botocore.config import Config are not guarded: if the SDK is missing/unimportable this raises an uncaught ImportError, exits non-zero, and writes NO note to PG_HISTORY_OUT, leaving the downstream "Upsert PR comment" step with empty output. The workflow's || true masks the exit code but not the missing note. Wrap the import in the same try/except-and-write-note pattern used for the MCP setup for consistency with the documented contract. (moderate confidence)

Suggested change
import boto3
from botocore.config import Config
try:
import boto3
from botocore.config import Config
except Exception as e:
open(OUT, "w").write(f"_pg-history: boto3/botocore unavailable: {e}_\n")
print(f"boto3 unavailable: {e}")
return

Comment thread .github/ocr/pg-history.py Outdated
Comment on lines +220 to +221
open(OUT, "w").write(body)
print(body)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

open(OUT, "w").write(...) leaves the file handle to be closed by refcount-based GC, so written bytes are only flushed at interpreter cleanup. This pattern is repeated at every write site (lines 158, 168, 172, 220). On CPython in this short-lived script it usually works, but it is not guaranteed on other interpreters and violates the tree's resource-management/portability expectations. Prefer with open(OUT, "w") as f: f.write(...). (low confidence, minor)

Comment thread .github/workflows/ocr-model-check.yml Outdated
Comment on lines +19 to +21
check-model:
runs-on: ubuntu-latest
steps:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This job has no timeout-minutes, unlike every job in .github/workflows/pg-ci.yml (which sets it on all jobs). The OIDC credential step and the inline Python subprocess invoking the AWS CLI can hang on network/throttling/OIDC issues, letting the job run to the default 6-hour runner limit. Add a short explicit timeout for this scheduled maintenance job.

Suggested change
check-model:
runs-on: ubuntu-latest
steps:
check-model:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:

Comment thread .github/workflows/sync-upstream.yml Outdated
Comment on lines +10 to +14
sync:
runs-on: ubuntu-latest
permissions:
contents: write
issues: write

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sync job has no timeout-minutes. git fetch upstream master against the full-history PostgreSQL repository plus a rebase can hang on network problems, and with an hourly schedule stalled runs can pile up and burn runner minutes indefinitely. Other workflows in this repo (pg-ci.yml) set timeout-minutes; add an explicit timeout here for consistency and safety.

Suggested change
sync:
runs-on: ubuntu-latest
permissions:
contents: write
issues: write
sync:
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: write
issues: write

Comment thread .github/workflows/sync-upstream.yml Outdated
echo "⚠️ Local master has $DIVERGED commits not in upstream"

# Check commit messages for "dev setup" or "dev v" pattern
DEV_SETUP_COMMITS=$(git log --format=%s upstream/master..origin/master | grep -iE "^dev (setup|v[0-9])" | wc -l)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The decision to rebase and force-push over master rests on brittle string matching: grep -iE "^dev (setup|v[0-9])" on commit subjects and grep -v "^\.github/" on changed paths. A squashed commit that mixes .github/ and non-.github/ changes, a merge commit, or a commit whose subject coincidentally starts with "dev v1" can be misclassified. A false positive here permits git rebase upstream/master followed by --force-with-lease, which can silently drop legitimate local commits on master. This is a data-loss footgun — the guard protecting destructive history rewriting should be more robust than subject-line pattern matching (e.g., fail closed on any non-.github/ change rather than trusting the commit subject).

Comment thread .github/workflows/sync-upstream.yml Outdated
Comment on lines +123 to +126
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"

if git rebase upstream/master; then

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

git config user.name/user.email is already set in the earlier "Configure Git" step, and git config persists across steps within a job on the runner. Re-setting it here is redundant and can be removed.

Suggested change
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
if git rebase upstream/master; then
if git rebase upstream/master; then

forknum = BufTagGetForkNum(&bufHdr->tag);
blocknum = bufHdr->tag.blockNum;
usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
usagecount = BUF_STATE_GET_COOLSTATE(buf_state);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change alters the user-visible semantics of the usage_count column exposed by all three functions (pg_buffercache_pages, pg_buffercache_summary, pg_buffercache_usage_counts). BUF_STATE_GET_USAGECOUNT returned the full 0..N usage count, whereas BUF_STATE_GET_COOLSTATE returns only the 0/1 cooling state (bit 0 of the field). The documentation in doc/src/sgml/pgbuffercache.sgml still describes and shows a multi-valued "usage count" (e.g. the sample output table and the column descriptions "A possible buffer usage count"), and is not updated in this change. A user-visible SQL-interface change without matching docs is incomplete for a pgsql-hackers patch. Update pgbuffercache.sgml to reflect the new HOT/COOL cooling-state semantics (and consider whether the exposed column/variable naming should still be usage_count). (high confidence)

Comment thread src/backend/access/heap/heapam.c Outdated
Comment on lines +398 to +401
allow_sync = (scan->rs_base.rs_flags & SO_ALLOW_SYNC) != 0;
}
else
allow_strat = allow_sync = false;

if (allow_strat)
{
/* During a rescan, keep the previous strategy object. */
if (scan->rs_strategy == NULL)
scan->rs_strategy = GetAccessStrategy(BAS_BULKREAD);
}
else
{
if (scan->rs_strategy != NULL)
FreeAccessStrategy(scan->rs_strategy);
scan->rs_strategy = NULL;
}
allow_sync = false;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This diff removes the bulk-read access strategy from initscan(), but the comment above this block (lines 383-390) still reads "use a bulk-read access strategy and enable synchronized scanning ... only two behaviors to tune rather than four ... some callers need to be able to disable one or both of these behaviors". After this change there is only one behavior left here (sync scan); the bulk-read strategy no longer exists. The comment now describes code that is gone and should be updated to mention only synchronized scanning, otherwise it will mislead readers into looking for strategy handling that isn't there. (moderate confidence)

@github-actions
github-actions Bot force-pushed the master branch 2 times, most recently from e495986 to d7cd5d6 Compare July 14, 2026 10:50
@gburd
gburd force-pushed the master branch 6 times, most recently from 576fa38 to 70ceeea Compare July 29, 2026 12:03

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 OCR found 26 issue(s).

  • 25 inline, 1 in summary (inline capped at 25)

📄 src/backend/storage/buffer/freelist.c (L24-L25)

Include is out of alphabetical order. PostgreSQL sorts includes alphabetically within a block; port/pg_numa.h belongs immediately after port/atomics.h (line 19), before the storage/* headers, not appended at the end. Reviewers on -hackers routinely flag this.

💡 Suggested change

Before:

#include "storage/subsystems.h"
#include "port/pg_numa.h"

After:

#include "storage/subsystems.h"

Comment thread .clangd
Comment on lines +1 to +2
Diagnostics:
MissingIncludes: None

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This entire file should not be part of the patch. PostgreSQL's own top-level .gitignore states the policy explicitly: "Auxiliary files from local workflows, your preferred editor, etc. should be ignored locally using $GIT_DIR/info/exclude or ~/.gitexclude." A .clangd file is precisely such an editor/tooling artifact (as are the companion .gdbinit and pg-aliases.sh files in this changeset). Committing personal IDE configuration is unrelated churn that will draw an immediate rejection on pgsql-hackers. Keep it in a local exclude instead of tracking it. [high confidence]

forknum = BufTagGetForkNum(&bufHdr->tag);
blocknum = bufHdr->tag.blockNum;
usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
usagecount = BUF_STATE_GET_COOLSTATE(buf_state);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correctness / backward-compat (high confidence): this silently changes the semantics of the user-visible usage_count column of pg_buffercache_pages. Previously it reported 0..BM_MAX_USAGE_COUNT (0..5); it now reports only the HOT/COOL bit (0 or 1), because BUF_STATE_GET_COOLSTATE masks bit 0 only and discards the second-chance ref bit (bit 1). Existing queries/monitoring that interpret usage_count as the old clock-sweep counter break with no error and no cue. A user-visible interface change like this needs the column renamed (e.g. cool_state) or at least documented in pg_buffercache.sgml, plus regression-test updates; none are in this patch.

{
buffers_used++;
usagecount_total += BUF_STATE_GET_USAGECOUNT(buf_state);
usagecount_total += BUF_STATE_GET_COOLSTATE(buf_state);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correctness (high confidence): usage_count_avg in pg_buffercache_summary now averages a 0/1 cooling bit instead of the 0..5 usage counter, so the column silently becomes "fraction of HOT buffers" while keeping its old name and documented meaning. This is a POLA violation for anyone consuming this column. Rename/redefine the column and update the docs, or keep the reported quantity consistent with its name.

CHECK_FOR_INTERRUPTS();

usage_count = BUF_STATE_GET_USAGECOUNT(buf_state);
usage_count = BUF_STATE_GET_COOLSTATE(buf_state);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consistency (moderate confidence): pg_buffercache_usage_counts now only ever emits rows for counts 0 and 1 (BM_MAX_USAGE_COUNT is redefined to BUF_COOLSTATE_HOT = 1), whereas the documented/expected behavior is one row per usage count 0..5. Array indexing is safe (2-element arrays sized by BM_MAX_USAGE_COUNT+1), but the SQL result shape changes silently. Update the function's docs and its regression test expected output, and consider whether the usage_count column here should be renamed to reflect the cooling-state semantics.

Comment thread .gdbinit
@@ -0,0 +1,156 @@
# HOT Indexed Updates — GDB breakpoints for code review

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This entire file is a personal developer debugging artifact, not part of any feature. A patch destined for pgsql-hackers/commitfest must be minimal and contain only what the change requires. Committing a top-level .gdbinit (alongside .clangd and pg-aliases.sh) pollutes the tree, makes the patch "do more than one thing," and is an immediate rejection reason. It also has security implications: a .gdbinit in a working directory auto-executes GDB commands, and shipping one in-tree is a footgun. Remove it from the patch and keep it local (e.g. via a personal, un-tracked gitignore).

Comment thread src/backend/storage/buffer/freelist.c Outdated
Comment on lines +195 to +197
SpinLockAcquire(&StrategyControl->buffer_strategy_lock);
StrategyControl->completePasses++;
SpinLockRelease(&StrategyControl->buffer_strategy_lock);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

completePasses is double-counted when batch_size > 1. When a fetch-add lands with start < NBuffers but the batch straddles the boundary (start + batch_size > NBuffers), this branch increments completePasses. But that same fetch-add pushes nextVictimBuffer past NBuffers, so the next backend to fetch-add gets start >= NBuffers and takes the wrap-CAS branch above, which increments completePasses again for the same physical wrap. Trace (NBuffers=100, batch=16): backend A start=90 -> straddle branch -> completePasses++ (counter now 106); backend B start=106 -> wrap branch -> CAS 106->6, completePasses++. One physical wrap, two increments. StrategySyncStart returns this to BgBufferSync, where an inflated passes_delta inflates strategy_delta (Assert(strategy_delta >= 0) and lap arithmetic), mispacing the LRU writer. The straddle-crossing is already implicitly accounted for by nextVictimBuffer / NBuffers in StrategySyncStart plus the eventual wrap-CAS, so this extra increment is redundant/wrong.


cmdline2 = psprintf("exec %s", cmdline);
execl(shellprog, shellprog, "-c", cmdline2, (char *) NULL);
execlp(shellprog, shellprog, "-c", cmdline2, (char *) NULL);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one-line change (execl -> execlp) is unrelated to the rest of the patch, which is a buffer-manager cooling-stage change, and does not belong in this commit. On -hackers a patch that bundles an unrelated change is a top rejection reason; drop this hunk (and the .clangd/.gdbinit/pg-aliases.sh dev artifacts) so the change reads as one logical unit.

Beyond patch hygiene, the switch is a behavioral loosening with no benefit for the in-tree use. shellprog is the compiled-in SHELLPROG, which is $(SHELL) from make or /bin/sh from meson -- an absolute path. execl invokes exactly that binary. execlp additionally performs a PATH search when the name contains no slash, so if SHELLPROG is ever a bare name the test harness will exec whatever sh sits earliest in the caller's PATH instead of the intended shell -- a reproducibility/security footgun in a test runner. There is no accompanying rationale, test, or commit-message justification. Revert to execl unless there is a documented need, in which case it should be a standalone patch. (high confidence)

Suggested change
execlp(shellprog, shellprog, "-c", cmdline2, (char *) NULL);
execl(shellprog, shellprog, "-c", cmdline2, (char *) NULL);

Comment thread src/include/storage/buf_internals.h Outdated
* max") reads naturally.
*/
#define BM_MAX_USAGE_COUNT 5
#define BM_MAX_USAGE_COUNT BUF_COOLSTATE_HOT

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BM_MAX_USAGE_COUNT is redefined from 5 to 1 while keeping the historical name. This is a backward-compatibility footgun: the macro is visible to extensions (it lives in buf_internals.h and is used by contrib/pg_buffercache, which sizes usage_counts[BM_MAX_USAGE_COUNT + 1] on it). Any out-of-tree consumer that still assumes the 0..5 range now silently gets a 2-element domain, with no compile-time signal. The macro name no longer describes anything ("max usage count" when there is no usage count), which the comment itself concedes ("reads naturally"). Prefer renaming to something that reflects the new meaning (e.g. reference the cooling state directly, BUF_COOLSTATE_HOT) at the call sites and dropping the misleading alias, rather than overloading a semi-public name with new semantics.

Comment thread src/include/storage/buf_internals.h Outdated
Comment on lines +139 to +140
/* Cooling state (HOT/COOL) from buffer state -- bit 0 of the field only, so
* the second-chance ref bit (bit 1) does not perturb the HOT/COOL test. */

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostgreSQL multi-line comment style puts the opening /* on its own line; here comment text starts on the same line as /*. This will not survive pgindent unchanged and is inconsistent with the surrounding comments in this file. s//* Cooling state...//*\n * Cooling state.../

Suggested change
/* Cooling state (HOT/COOL) from buffer state -- bit 0 of the field only, so
* the second-chance ref bit (bit 1) does not perturb the HOT/COOL test. */
/*
* Cooling state (HOT/COOL) from buffer state -- bit 0 of the field only, so
* the second-chance ref bit (bit 1) does not perturb the HOT/COOL test.
*/

@@ -1,4 +1,4 @@
#!/usr/bin/perl
#!/usr/bin/env perl

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shebang change breaks tree-wide consistency and is not minimal. Every other Perl script in the tree uses #!/usr/bin/perl (genbki.pl, gen_node_support.pl, copyright.pl, mark_pgdllimport.pl, and ~30 others). Changing only pgindent to /usr/bin/env perl is an unrelated, standalone edit with no functional change requiring it, so it will be flagged as churn on pgsql-hackers. It also weakens reproducibility: /usr/bin/env perl picks up whatever perl is first on PATH (e.g. a perlbrew/plenv build) rather than the system perl the tree standardizes on, which can change pgindent's output and is a POLA violation for a formatting tool. Revert to the canonical shebang.

Suggested change
#!/usr/bin/env perl
#!/usr/bin/perl

@gburd
gburd force-pushed the master branch 3 times, most recently from 2746528 to 3f6fff3 Compare July 29, 2026 20:59

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 OCR found 28 issue(s).

  • 25 inline, 3 in summary (inline capped at 25)

📄 src/include/storage/buf_internals.h (L141-L142)

BUF_STATE_GET_USAGECOUNT is now dead: after this change every caller reads BUF_STATE_GET_COOLSTATE instead, and a codebase-wide search finds no remaining users of BUF_STATE_GET_USAGECOUNT (only this definition). Since the accessor no longer reflects a real field (the former usagecount is now a 1-bit cool state + 1-bit ref bit), leaving it in place is a footgun: a future caller could resurrect it expecting the old 0..5 count and instead get the raw 2-bit cool|ref field (0..3). Either remove it or repoint it explicitly. Confidence: high.


📄 src/include/storage/buf_internals.h (L139-L140)

Multi-line block comments in this file (and the tree convention) put /* alone on its own line. This opens with text on the /* line. Minor style nit; align with the surrounding comments in this header. Confidence: medium.

💡 Suggested change

Before:

+/* Cooling state (HOT/COOL) from buffer state -- bit 0 of the field only, so
+ * the second-chance ref bit (bit 1) does not perturb the HOT/COOL test. */

After:

+/*
+ * Cooling state (HOT/COOL) from buffer state -- bit 0 of the field only, so
+ * the second-chance ref bit (bit 1) does not perturb the HOT/COOL test.
+ */

📄 src/include/storage/buf_internals.h (L191-L191)

Retaining the name BM_MAX_USAGE_COUNT for a value that is now the single-bit HOT constant (1) is a POLA violation: a maintainer reading usage_counts[BM_MAX_USAGE_COUNT + 1] in pg_buffercache, or a loop bounded by it, will assume a 0..5 multi-value count and be misled. The pin fast path already uses BUF_COOLSTATE_HOT directly, so the "reads naturally" justification is thin. Prefer removing this alias and using BUF_COOLSTATE_HOT at the (few) sites, or renaming, to avoid a lingering misnomer in the core buffer manager. Confidence: medium.

Comment thread .clangd
Comment on lines +1 to +2
Diagnostics:
MissingIncludes: None

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This .clangd file is personal, developer-local editor tooling that must not be part of a PostgreSQL patch. It is unrelated to the actual change (the HOT indexed updates / buffer manager work in the other files) and does more than the patch's stated purpose. A few concrete problems:

  1. It is not minimal-diff: adding a per-developer LSP config alongside .gdbinit and pg-aliases.sh is exactly the kind of unrelated scaffolding that gets a patch rejected on -hackers. The tree has no .clangd, and adding one imposes one contributor's editor setup on everyone.
  2. It hardcodes a relative include path -I../../../../src/include, which only resolves from a specific nested subdirectory and is meaningless at the repository root where this file lives.
  3. -DPGDLLIMPORT= silently blanks out PGDLLIMPORT, which would hide exactly the Windows/MSVC export-annotation issues the project cares about (a footgun for anyone relying on clangd diagnostics).

If local clangd config is desired, it belongs in the developer's own environment (or .git/info/exclude / a personal gitignore), not committed to the tree. Recommend dropping this file (and .gdbinit, pg-aliases.sh) from the patch entirely.

forknum = BufTagGetForkNum(&bufHdr->tag);
blocknum = bufHdr->tag.blockNum;
usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
usagecount = BUF_STATE_GET_COOLSTATE(buf_state);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This silently repurposes a user-visible column. pg_buffercache_pages still exposes this value through the usage_count int2 column (its name and docs unchanged), but it will now report a 0/1 cooling-state bit instead of the documented 0..5 clock-sweep usage count. That is a backward-incompatible semantic change to a stable extension view with no accompanying pg_buffercache SQL/version bump or doc update, so existing monitoring queries silently break. Either keep the column reporting a true count, or rename/redocument the column (new extension version) and rename the misleading local usagecount variable to reflect that it now holds a HOT/COOL state. high confidence.

{
buffers_used++;
usagecount_total += BUF_STATE_GET_USAGECOUNT(buf_state);
usagecount_total += BUF_STATE_GET_COOLSTATE(buf_state);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

usagecount_total now accumulates the 0/1 cooling-state bit, so the view's usagecount_avg column (name/docs unchanged) silently changes meaning from an average 0..5 usage count to an average HOT/COOL bit. Same backward-compatibility concern as in pg_buffercache_pages: needs a column rename + doc/version update, not a silent substitution. high confidence.

CHECK_FOR_INTERRUPTS();

usage_count = BUF_STATE_GET_USAGECOUNT(buf_state);
usage_count = BUF_STATE_GET_COOLSTATE(buf_state);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pg_buffercache_usage_counts groups rows by usage_count (0..5). With BM_MAX_USAGE_COUNT now 1, this SRF collapses to just two rows (0 and 1) whose meaning is a HOT/COOL bit, not a usage count, while the output column is still named usage_count. This is a user-visible, backward-incompatible change to the view output with no doc/SQL update and a now-misleading usage_count local variable name. high confidence.

Comment thread .gdbinit
Comment on lines +1 to +4
# HOT Indexed Updates — GDB breakpoints for code review
#
# Usage: gdb -x .gdbinit <postgres-binary>
# Or from gdb: source .gdbinit

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This .gdbinit is a personal debugging artifact and should not be part of a PostgreSQL contribution. It is not referenced by any build/test/gitignore infrastructure (confirmed: no references anywhere in the tree), is required by none of the actually-changed files (bufmgr/freelist/localbuf/pg_buffercache/pg_regress/pgindent/.clangd/pg-aliases.sh are all unrelated to HOT indexed updates), and placing an auto-loading .gdbinit at the repo root is a footgun: any developer running gdb in this directory silently gets these breakpoints (and gdb may refuse to auto-load or warn, depending on auto-load safe-path). Drop this file from the patch entirely.

Severity: high confidence.


cmdline2 = psprintf("exec %s", cmdline);
execl(shellprog, shellprog, "-c", cmdline2, (char *) NULL);
execlp(shellprog, shellprog, "-c", cmdline2, (char *) NULL);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This execl -> execlp change is unrelated to the rest of this changeset (buffer-manager work in bufmgr.c/freelist.c/localbuf.c) and does not belong in this patch. Split it out per minimal-diff discipline.

More importantly, it is a functional no-op as written: shellprog is SHELLPROG, defined as $(SHELL) in GNUmakefile and hardcoded to /bin/sh in meson.build -- always an absolute path. execlp only performs a PATH search when the program name contains no slash, so with an absolute path it behaves identically to execl. The change buys nothing.

If the intent is to let shellprog be resolved via PATH, that is a real behavioral change (and a security footgun -- searching PATH for the shell that runs test commands) that needs justification, tests, and a matching change to how SHELLPROG is defined. Otherwise revert this hunk. (high confidence)

Suggested change
execlp(shellprog, shellprog, "-c", cmdline2, (char *) NULL);
execl(shellprog, shellprog, "-c", cmdline2, (char *) NULL);

Comment thread src/backend/storage/buffer/freelist.c Outdated
Comment on lines +170 to +173
current = pg_atomic_read_u32(&StrategyControl->nextVictimBuffer);
if (current >= (uint32) NBuffers)
{
wrapped = current % NBuffers;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With batching, nextVictimBuffer is only wrapped opportunistically: the CAS uses the re-read current, and racing backends can leave the shared counter well above NBuffers (2*NBuffers or more). StrategySyncStart() then computes *complete_passes += nextVictimBuffer / NBuffers, which is no longer bounded by 1 as the stock code assumes, and StrategySyncStart's result = nextVictimBuffer % NBuffers is the only thing keeping the returned index in range. Combined with the double-increment above, the completed-pass count reported to BgBufferSync becomes unreliable. The invariant "nextVictimBuffer stays within [0, NBuffers) except for a transient single wrap" is broken; the wrap logic needs to guarantee the counter is brought back below NBuffers deterministically before other backends advance it further.

Comment thread src/backend/storage/buffer/freelist.c Outdated
#include "storage/proc.h"
#include "storage/shmem.h"
#include "storage/subsystems.h"
#include "port/pg_numa.h"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Include ordering: PostgreSQL groups and alphabetizes headers within a block. port/pg_numa.h is appended after the storage/* includes; port/ sorts before storage/. Move it up to keep the block ordered and pgindent/reviewer-clean.

Comment thread src/backend/storage/buffer/freelist.c Outdated
* grace before eviction.
*/
if (BUF_STATE_GET_REFBIT(local_buf_state))
local_buf_state &= ~BUF_REFBIT; /* second chance: clear ref, stay HOT */

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These column-aligned trailing comments on the assignment statements are likely to be reflowed by pgindent (which manages tab alignment) and produce spurious diff churn / possible trailing-whitespace. Prefer a single explanatory comment above the if/else rather than aligned inline comments on each branch.

Comment thread src/include/storage/buf_internals.h Outdated
Comment on lines +92 to +100
* position above the cooling-state bit. PinBuffer sets it on every access.
* The bgwriter's pre-cooling gives a HOT buffer a second chance: the first
* time it passes a HOT buffer whose ref bit is set, it clears the ref bit and
* leaves the buffer HOT; only a HOT buffer whose ref bit is already clear (not
* re-accessed since the previous bgwriter pass) is demoted to COOL. This
* keeps genuinely-hot pages out of the COOL stage (protecting the working set
* from being cooled under scan pressure) while leaving the foreground sweep a
* single-pass search over the pre-staged COOL buffers. A separate bit (not a
* count) so it stays a plain masked store under the header lock.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment misdescribes the shipped behavior (stale/aspirational design). It states the bgwriter's pre-cooling applies the second chance and pre-stages COOL buffers for a "single-pass" foreground sweep. But the shipped code does the opposite:

  • The foreground sweep in freelist.c is a "COOL-IN-PLACE" sweep that demotes HOT->COOL on every tick and applies the second-chance ref-bit logic itself (freelist.c: if (BUF_STATE_GET_REFBIT(...)) local_buf_state &= ~BUF_REFBIT;), not the bgwriter.
  • BgBufferSync calls SyncOneBuffer(next_to_clean, true, false, wb_context) with cool_if_hot = false, and its own comment says the bgwriter does "no background pre-cooling" because the sweep handles all cooling.

So the described bgwriter pre-cooling path is not exercised, and this header comment attributes the second-chance mechanism to the wrong actor. Rewrite it to describe what the foreground sweep actually does (per the review standard that comments describe current behavior, not a superseded design). Confidence: high.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 OCR found 31 issue(s).

  • 25 inline, 6 in summary (inline capped at 25)

📄 src/tools/pgindent/pgindent (L1-L1)

This shebang change diverges from the project-wide convention: every other Perl script in the tree (genbki.pl, gen_node_support.pl, copyright.pl, mark_pgdllimport.pl, etc.) uses #!/usr/bin/perl. Switching only pgindent to #!/usr/bin/env perl is an unrelated, inconsistent change that does not belong in a functional patch. Such a convention change would need its own -hackers discussion applied tree-wide, not a one-off here. Revert to keep the diff minimal and consistent.

💡 Suggested change

Before:

#!/usr/bin/env perl

After:

#!/usr/bin/perl

📄 src/include/storage/buf_internals.h (L100-L102)

The atomicity claim here is inaccurate for the primary hot path. PinBuffer() in bufmgr.c sets BUF_REFBIT inside a lock-free CAS loop (pg_atomic_compare_exchange_u64) precisely because it is not allowed to touch the refcount while holding the header spinlock (see the comment at bufmgr.c: "We're not allowed to increase the refcount while the buffer header spinlock is held"). The ref bit is only cleared under the header lock (SyncOneBuffer via UnlockBufHdrExt). Stating it "stays a plain masked store under the header lock" misdescribes the mechanism and is a footgun: a future reader may assume all BUF_REFBIT mutations are lock-protected and drop the CAS. Reword to reflect that the set path is a CAS-loop store folded into the existing pin CAS, and only the clear path runs under the header lock. (high confidence)


📄 src/backend/storage/buffer/freelist.c (L195-L197)

completePasses accounting is inconsistent with StrategySyncStart under batching, and drifts in both directions.

StrategySyncStart() returns completePasses + nextVictimBuffer / NBuffers. The stock invariant is that every time the counter crosses a multiple of NBuffers, exactly one pass is folded into completePasses. With batching two problems appear:

  1. Undercount: a single fetch-add of batch_size can push nextVictimBuffer across the boundary while landing >= NBuffers with current / NBuffers > 1; the wrap path folds in only completePasses++ (one pass) and sets wrapped = current % NBuffers, discarding the extra whole passes.

  2. Double relative to the transient term: this else if branch increments completePasses for a sub-NBuffers batch that straddles the wrap, but it leaves nextVictimBuffer un-wrapped (>= NBuffers). StrategySyncStart then ALSO adds nextVictimBuffer / NBuffers for the same crossing, counting it twice until the next >=NBuffers fetch-add wraps the counter.

The comment concedes this feeds only bgwriter pacing, but the drift is unbounded (grows with concurrency x batch), which can materially misestimate BgBufferSync's scan-ahead. At minimum this needs the fold to account for current / NBuffers passes and the two paths to be reconciled with the StrategySyncStart transient term. (high confidence)


📄 src/backend/storage/buffer/freelist.c (L24-L25)

Include is out of order. The storage/* group is alphabetized (buf_internals, bufmgr, proc, shmem, subsystems); port/pg_numa.h belongs in the port/ group right after port/atomics.h, not appended at the end. pgindent won't fix include ordering, so a committer will bounce this.

💡 Suggested change

Before:

#include "storage/subsystems.h"
#include "port/pg_numa.h"

After:

#include "port/atomics.h"
#include "port/pg_numa.h"
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
#include "storage/proc.h"
#include "storage/shmem.h"
#include "storage/subsystems.h"

📄 src/backend/storage/buffer/freelist.c (L496-L499)

This diff bundles two independent, user-visible changes: (1) NUMA-gated batched clock sweep and (2) a full replacement-policy rewrite from multi-value usage_count to a 1-bit cool-state (BUF_COOLSTATE/BUF_REFBIT). Per pgsql-hackers norms these must be split into separate, independently-committable patches. Neither ships regression/TAP tests or docs, and both change buffer-eviction behavior. Additionally the changeset carries developer-environment cruft (.clangd, .gdbinit, flake.nix, shell.nix, pg-aliases.sh) that must not be part of the patch. As-is this draws immediate rejection on the list. (high confidence)


📄 src/backend/storage/buffer/freelist.c (L288-L291)

Comment overstates the shipped design and contradicts the sibling changes. It claims the sweep "never needs a background pre-cooler" and "cannot get stuck in a 'no COOL victim anywhere' state that forces a foreground escalation pass." But bufmgr.c's SyncOneBuffer() still implements a cool_if_hot pre-cooling path, and buf_internals.h's BUF_REFBIT comment describes "the bgwriter's pre-cooling gives a HOT buffer a second chance." Even though BgBufferSync currently calls SyncOneBuffer(..., cool_if_hot=false), the dead pre-cooling machinery plus these mutually contradictory comments will confuse reviewers. Either wire it in or remove it, and make the comments describe what the code does now. (moderate confidence)

Comment thread .clangd
Comment on lines +1 to +2
Diagnostics:
MissingIncludes: None

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This .clangd file should not be committed. It is an editor/language-server auxiliary file, and the repository's top-level .gitignore explicitly documents the policy: "Auxiliary files from local workflows, your preferred editor, etc. should be ignored locally using $GIT_DIR/info/exclude or ~/.gitexclude." This change is also unrelated to the buffer-manager work in the rest of the patch (bufmgr.c/freelist.c/localbuf.c) and bundles personal tooling into a functional patch. A patch posted to pgsql-hackers containing personal IDE config will be rejected as out-of-scope. Remove it from the commit and exclude it locally instead.

Comment thread .clangd
Comment on lines +42 to +43
- -I.
- -I../../../../src/include

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The include path -I../../../../src/include is hardcoded to a relative depth that only resolves when clangd is invoked from one specific subdirectory nesting (four levels below the tree root). Combined with CompilationDatabase: build/, this reflects a single developer's local layout and won't work generally. This further confirms the file is a personal, non-portable local artifact that does not belong in the shared repository.

forknum = BufTagGetForkNum(&bufHdr->tag);
blocknum = bufHdr->tag.blockNum;
usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
usagecount = BUF_STATE_GET_COOLSTATE(buf_state);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a user-visible behavior change that leaks a private buffer-manager rename into a monitoring extension's public API. usage_count/usagecount here now only ever reports 0 (COOL) or 1 (HOT) instead of the documented 0..5 range, while the SQL-visible column names (usage_count int4 in pg_buffercache--1.3--1.4.sql) and the pg_buffercache.pages usage_count column are unchanged. Consumers filtering/grouping on usage_count (e.g. ... WHERE usage_count = 5) silently break.

Also, the semantics no longer match the name: BUF_STATE_GET_COOLSTATE returns a HOT/COOL bit, not a usage count. Wiring it into a variable/column still called usagecount is misleading (POLA).

At minimum this change needs: (1) doc updates in contrib/pg_buffercache's SGML describing the new range/meaning, (2) updated column/naming so the exposed value is not called usage_count if it is now a cooling bit, and (3) this should not ride along silently with the buf_internals change -- it is a separate, documented, user-facing decision. Tag: high confidence on the behavior/doc drift.

CHECK_FOR_INTERRUPTS();

usage_count = BUF_STATE_GET_USAGECOUNT(buf_state);
usage_count = BUF_STATE_GET_COOLSTATE(buf_state);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In pg_buffercache_usage_counts the array bounds happen to stay safe (BM_MAX_USAGE_COUNT is now BUF_COOLSTATE_HOT == 1, so usage_counts/dirty/pinned are size 2 and BUF_STATE_GET_COOLSTATE returns 0 or 1). But the emitted result set now has only two rows (usage_count 0 and 1) where callers expect six (0..5). The existing regress test only asserts count(*) > 0, so this contract change is untested. Add a regression test that pins the expected row count/values, and update the docs. Tag: high confidence.

Comment thread flake.nix
inherit (shellConfig) gdbConfig flameGraphScript pgbenchScript;
};

environment.localBinInPath = true;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

environment.localBinInPath is a NixOS module option, not a valid flake output. Because it sits inside the eachDefaultSystem closure, it becomes environment.<system>.localBinInPath in the flake outputs, which the flake schema does not recognize and which has no effect on the dev shells. It is dead config that should be removed (the option only makes sense inside a NixOS configuration.nix / module, not here).

Comment thread src/backend/storage/buffer/bufmgr.c Outdated
Comment on lines +4095 to +4097
write_limit = bgwriter_lru_maxpages;
if (upcoming_alloc_est > write_limit)
write_limit = upcoming_alloc_est;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This silently overrides the user-facing GUC bgwriter_lru_maxpages, which is documented as the maximum number of pages the bgwriter may write per round. Raising the cap to upcoming_alloc_est violates POLA and the GUC's contract, changes the observable I/O footprint under bulk workloads, and ships with no doc update, no benchmark, and no design-thread reference. Note also that maxwritten_clean is incremented at line ~4127 when this raised limit is reached, mislabeling the stat that is meant to count hits of the maxpages cap. If ahead-of-demand cleaning is desired, this needs to be justified as a behavior change (or gated), not folded in silently. Moderate confidence.

Comment thread src/backend/storage/buffer/bufmgr.c Outdated
* a shorter window than the strategy proxy, tracking a burst of
* probationary/scan COOL pages within a cycle or two instead of lagging it.
*/
float cleaner_smoothing_samples = 4;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cleaner_smoothing_samples = 4 is an unexplained magic constant that changes long-standing bgwriter LRU pacing (the original used smoothing_samples = 16 here). No GUC, no benchmark, no design-discussion reference for the 16 -> 4 change in adaptivity. Justify the value or make it a named/derived constant with rationale; a large behavioral tuning change should not ride in on a bare literal. Moderate confidence.

Comment thread src/backend/storage/buffer/bufmgr.c Outdated
Comment on lines +2337 to +2340
set_bits |= BM_TAG_VALID;
/* Admit the newly loaded page COOL (probation); a second access via
* PinBuffer promotes it to HOT. This is what makes a one-touch scan
* self-evicting -- see the cooling-state notes in buf_internals.h. */

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostgreSQL block comments start with the text on the line after the opening /* (see the surrounding house style). This same-line, multi-line form and the paragraph-length narration of the algorithm will not survive pgindent cleanly and describe WHAT more than WHY. Reflow to standard style and trim to the rationale. Also verify -- here and elsewhere in the new comments is ASCII hyphen-minus, not a smart/em dash (source must be ASCII-only). Low confidence.

Suggested change
set_bits |= BM_TAG_VALID;
/* Admit the newly loaded page COOL (probation); a second access via
* PinBuffer promotes it to HOT. This is what makes a one-touch scan
* self-evicting -- see the cooling-state notes in buf_internals.h. */
set_bits |= BM_TAG_VALID;
/*
* Admit the newly loaded page COOL (probation); a second access via
* PinBuffer promotes it to HOT. This is what makes a one-touch scan
* self-evicting; see the cooling-state notes in buf_internals.h.
*/

Comment thread src/backend/storage/buffer/localbuf.c Outdated
if (BUF_STATE_GET_COOLSTATE(buf_state) != BUF_COOLSTATE_COOL)
{
buf_state -= BUF_USAGECOUNT_ONE;
/* HOT: give it a second chance, cool it and keep scanning. */

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment "give it a second chance" is inaccurate for the local sweep and drifts from what the code does. In the shared-buffer sweep (freelist.c) the "second chance" is the second-chance reference bit (BUF_REFBIT): a HOT buffer with the ref bit set is spared (ref bit cleared, stays HOT) and only demoted on the next pass. Here there is no ref-bit logic at all: a HOT local buffer is demoted HOT -> COOL unconditionally on the first pass this sweep passes it. Local buffers never set BUF_REFBIT (PinLocalBuffer does not set it), so the reprieve the phrase implies does not exist. Reword to describe the actual behavior, e.g. "HOT: demote to COOL and keep scanning; it becomes a candidate on the next pass." (moderate confidence)

Suggested change
/* HOT: give it a second chance, cool it and keep scanning. */
/* HOT: demote to COOL and keep scanning; a later pass may evict it. */


cmdline2 = psprintf("exec %s", cmdline);
execl(shellprog, shellprog, "-c", cmdline2, (char *) NULL);
execlp(shellprog, shellprog, "-c", cmdline2, (char *) NULL);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This execl -> execlp change is a behavioral change with no visible justification, no comment, and no test. In practice shellprog is SHELLPROG, i.e. /bin/sh (meson.build) or $(SHELL) (GNUmakefile) — normally an absolute path. Per POSIX, execlp performs a PATH search only when the file argument contains no slash; with a slash it behaves exactly like execl. So for standard builds this is a no-op, and the only case it changes is when shellprog has no directory component, where it now resolves the shell via the runtime PATH.

Two problems for a pgsql-hackers patch:

  1. The rationale is absent. If the intent is to support a $(SHELL) value without a directory component, state it in the commit message and add a comment here; otherwise this reads as an unrelated, unexplained diff (minimal-diff discipline). Confidence: high.
  2. Resolving the interpreter through PATH instead of a build-configured absolute path is a footgun (non-deterministic/less secure resolution). If that is not the goal, keep execl. Confidence: moderate.

Either add justification + a comment (and ideally a test exercising the intended new behavior), or revert to execl.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 OCR found 40 issue(s).

  • 25 inline, 15 in summary (inline capped at 25)

📄 pg-aliases.sh (L417-L417)

The regression port is hardcoded to --port=40099 with no override. Two concurrent pg-test-one runs, or a leftover backend still bound to 40099, will collide and fail non-obviously. PostgreSQL's own test harness derives a unique port; at minimum make this configurable (e.g. --port="${PG_TEST_PORT:-40099}") or pick a random free port.


📄 src/tools/pgindent/pgindent (L1-L1)

This shebang change is unrelated to the PR's stated purpose (buffer-manager work) and breaks a tree-wide convention: every other Perl script in the source tree uses #!/usr/bin/perl (verified: genbki.pl, gen_node_support.pl, copyright.pl, mark_pgdllimport.pl, and ~30 others). Making pgindent the sole #!/usr/bin/env perl outlier is inconsistent, and env perl performs a PATH search that resolves to whatever perl happens to be first on PATH (a footgun in mixed environments / with local perlbrew installs). This is a local-convenience change that does not belong in a patch destined for -hackers; drop it to keep the diff minimal. (high confidence)

💡 Suggested change

Before:

#!/usr/bin/env perl

After:

#!/usr/bin/perl

📄 src/test/regress/pg_regress.c (L1246-L1246)

Switching execl -> execlp is unrelated to the PR's stated purpose and changes behavior subtly: execlp performs a PATH search when the program name contains no slash. With the default SHELLPROG ($(SHELL) via autoconf, "/bin/sh" via meson) the value is an absolute path, so today this is behavior-neutral; but if $(SHELL) were ever a bare name, the shell would be resolved from PATH instead of the configured path -- a non-obvious footgun in a test harness that should run a deterministic shell. There is no justification in this PR for the change. Recommend reverting to keep the diff minimal, or, if intentional, splitting it into its own commit with a rationale. (moderate confidence)

💡 Suggested change

Before:

		execlp(shellprog, shellprog, "-c", cmdline2, (char *) NULL);

After:

		execl(shellprog, shellprog, "-c", cmdline2, (char *) NULL);

📄 pg-aliases.sh (L508-L508)

modified_files is assigned without local, so it leaks into the interactive shell that sourced pg-aliases.sh and persists after pg-format returns. The sibling helpers pg-tidy and pg-spell correctly declare local files. Declare it local for consistency and to avoid polluting the caller's environment.

💡 Suggested change

Before:

		modified_files=$(git diff --name-only "${since}" | grep -E "\.c$|\.h$")

After:

		local modified_files
		modified_files=$(git diff --name-only "${since}" | grep -E "\.c$|\.h$")

📄 shell.nix (L389-L389)

TRANSACTIONS ($3) is captured, printed in the config banner and written into the results header, but it is never passed to pgbench: the run is driven purely by -T "$DURATION" (time-based). A user who passes a transaction count (e.g. via pg-bench-run 10 2 5000 ...) will see it echoed but it has no effect on the benchmark. Either pass it through with -t "$TRANSACTIONS" (mutually exclusive with -T) or drop the parameter to avoid the misleading knob.


📄 src/backend/storage/buffer/bufmgr.c (L4231-L4248)

Dead code: cool_if_hot is passed false from both call sites (BufferSync line 3790, BgBufferSync line 4112), so this entire pre-cooling block never executes. BUF_COOLED is set here but never read by any caller (callers only test BUF_WRITTEN/BUF_REUSABLE). Per the minimalism/YAGNI discipline, drop the cool_if_hot parameter, this block, and the BUF_COOLED flag entirely, or wire up a caller that actually uses them. Shipping the unused path also carries the unlock/re-lock overhead in the signature and confuses readers about which cooling strategy is live. (high confidence)


📄 src/backend/storage/buffer/bufmgr.c (L3345-L3347)

Stale comment: it claims the ref bit lets "the bgwriter's next cooling pass spare this recently-used buffer", but the bgwriter never runs a cooling pass -- both SyncOneBuffer call sites pass cool_if_hot=false (see the comment at line 4107). The ref bit is actually consumed only by the foreground clock sweep in StrategyGetBuffer(). Update the comment to describe the live behavior. (medium confidence)


📄 src/backend/storage/buffer/bufmgr.c (L4095-L4097)

This silently overrides the documented meaning of bgwriter_lru_maxpages. config.sgml states "no more than this many buffers will be written by the background writer" per round; raising write_limit to upcoming_alloc_est breaks that contract under bulk-dirtying. A user who lowered the GUC to intentionally leave writes to backends no longer gets the cap they set. This is a user-visible GUC-semantics change shipped with no doc update and no regression test -- WIP by pgsql-hackers standards. Either document the new behavior (and ideally gate it behind an opt-in) or drop the override. (medium confidence)


📄 src/backend/storage/buffer/freelist.c (L195-L197)

completePasses can be over-counted per sweep pass. The start >= NBuffers branch increments it at most once (via CAS), but this else if branch increments unconditionally for every backend whose batch straddles the wrap point. With batching and many concurrent backends, several batches can straddle NBuffers in the same pass (and this can co-occur with the wrap-CAS increment), so completePasses can advance by more than one per real pass. StrategySyncStart also adds nextVictimBuffer/NBuffers on top. Since completePasses feeds bgwriter pacing, confirm the resulting skew is acceptable, or count the wrap crossing exactly once. (medium confidence)


📄 src/backend/storage/buffer/freelist.c (L24-L25)

Include is out of order. PostgreSQL sorts includes alphabetically within the group; "port/pg_numa.h" belongs right after "port/atomics.h", not after the storage/* block. pgindent/committer style will flag this. (low confidence)

💡 Suggested change

Before:

#include "storage/subsystems.h"
#include "port/pg_numa.h"

After:

#include "port/atomics.h"
#include "port/pg_numa.h"
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
#include "storage/proc.h"
#include "storage/shmem.h"
#include "storage/subsystems.h"

📄 src/include/storage/buf_internals.h (L93-L96)

Stale comment: it describes "the bgwriter's pre-cooling" giving a HOT buffer a second chance, but the bgwriter's SyncOneBuffer calls all pass cool_if_hot=false, so no bgwriter pre-cooling pass exists. The second-chance ref bit is consumed by the foreground clock sweep in StrategyGetBuffer(). Rewrite so the comment describes the actual (foreground-sweep) behavior rather than the dead bgwriter path. (medium confidence)


📄 contrib/pg_buffercache/pg_buffercache_pages.c (L164-L164)

User-visible semantic change without a doc/version update. This column previously reported the 0..5 usage_count; it now reports the 0/1 cooling state. pg_buffercache_usage_counts() will now only emit rows for usage_count 0 and 1 (the [BM_MAX_USAGE_COUNT+1] arrays are correctly resized to [2], so no OOB), and pg_buffercache.usagecount changes meaning. The pg_buffercache documentation (pgbuffercache.sgml) still describes a 0..5 usage count and needs updating; the expected regression output (expected/pg_buffercache*.out) will also change. Ship the doc/test updates with this change. (medium confidence)


📄 src/backend/storage/buffer/bufmgr.c (L2338-L2340)

Non-standard block comment format: PostgreSQL style (and pgindent) require the opening /* to be alone on its own line for a multi-line block comment. Text on the same line as /* will be reflowed by pgindent, creating diff churn against the committed tree. Same issue at the ExtendBufferedRelShared admission site below. (low confidence / style)

💡 Suggested change

Before:

	/* Admit the newly loaded page COOL (probation); a second access via
	 * PinBuffer promotes it to HOT.  This is what makes a one-touch scan
	 * self-evicting -- see the cooling-state notes in buf_internals.h. */

After:

	/*
	 * Admit the newly loaded page COOL (probation); a second access via
	 * PinBuffer promotes it to HOT.  This is what makes a one-touch scan
	 * self-evicting -- see the cooling-state notes in buf_internals.h.
	 */

📄 src/backend/storage/buffer/bufmgr.c (L3010-L3011)

Same non-standard block comment format as the other admission site: the opening /* must be alone on its own line. pgindent will reformat this. (low confidence / style)

💡 Suggested change

Before:

			/* Admit COOL (probation); see the comment at the other admission
			 * site and the cooling-state notes in buf_internals.h. */

After:

			/*
			 * Admit COOL (probation); see the comment at the other admission
			 * site and the cooling-state notes in buf_internals.h.
			 */

📄 src/backend/storage/buffer/localbuf.c (L253-L254)

Comment is inaccurate for the local-buffer path. Unlike the shared clock sweep in StrategyGetBuffer(), this path implements no second-chance reference bit: PinLocalBuffer() never sets BUF_REFBIT, and this branch unconditionally demotes HOT -> COOL via buf_state -= BUF_COOLSTATE_ONE. There is no "second chance" here. The comment should describe the actual single-bit HOT->COOL demotion. (moderate confidence)

💡 Suggested change

Before:

				/* HOT: give it a second chance, cool it and keep scanning. */
				buf_state -= BUF_COOLSTATE_ONE;

After:

				/* HOT: demote to COOL and keep scanning. */
				buf_state -= BUF_COOLSTATE_ONE;

Comment thread .gdbinit
@@ -0,0 +1,156 @@
# HOT Indexed Updates — GDB breakpoints for code review

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This .gdbinit is a personal debugger artifact that must not be committed. The repository's own root .gitignore (lines 1-3) states policy explicitly: "Auxiliary files from local workflows, your preferred editor, etc. should be ignored locally using $GIT_DIR/info/exclude or ~/.gitexclude." A debugger config for one developer's session belongs in $GIT_DIR/info/exclude, not in a patch destined for pgsql-hackers/commitfest.

Beyond scope, this is a security footgun: GDB auto-loads .gdbinit from the current working directory, so any developer who runs gdb from the repo root silently executes its contents -- a well-known code-execution vector. It also defeats the auto-load safe-path protections GDB ships to warn against exactly this. [high confidence] Recommend removing this file from the change entirely.

Comment thread .gdbinit
# =========================================================================

# Main entry: heap_update
break heapam.c:3210

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hardcoded source line breakpoints do not match the current tree and reference nonexistent code. Verified against the repo:

  • break heapam.c:3210 is a comment for simple_heap_delete, not heap_update (the file has 9524 lines; the comment here is wrong).
  • break heapam.c:4019/4024/4033 point to the deadlock-retry / no-TOAST branch of the existing update path, not any "pure HOT vs HOT indexed" decision.
  • break heapam_indexscan.c:250/297 and :182 reference HOT-indexed accumulator logic, but that file is only 299 lines and contains no such code.

Furthermore, the function-name breakpoints (heap_hot_indexed_create_tuple, heap_hot_indexed_serialize_bitmap, heap_hot_indexed_read_bitmap, heap_xlog_indexed_update, etc.) and the HEAP_INDEXED_UPDATED flag do not exist anywhere in the tree (code_search returned no matches). The entire "HOT indexed updates" series this file debugs is absent from both this review group and the other-changed-files list. This makes the file dead/misleading on its own commit and violates the atomic/bisectable-commit requirement: it documents and depends on code that was never added. [high confidence]

Comment thread .clangd
ParameterNames: true
DeducedTypes: true
CompileFlags:
CompilationDatabase: build/ # Search build/ directory for compile_commands.json

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This .clangd LSP config is a personal editor artifact and should not be committed, per the repository's own root .gitignore policy (lines 1-3: "Auxiliary files from ... your preferred editor ... should be ignored locally using $GIT_DIR/info/exclude or ~/.gitexclude"). It is out of scope for a pgsql-hackers/commitfest patch and pollutes the diff. [high confidence]

Additionally the config is not portable to other contributors: CompilationDatabase: build/ and -I../../../../src/include encode one developer's build layout and assumed working directory (no build/ compilation database exists or is generated at a standard tracked location in this tree). As a repo-root config it silently applies -DDEBUG, -DLOCAL, -DPGDLLIMPORT= and other flags to every file opened in any clangd-enabled editor, which can mask real diagnostics (e.g. defining PGDLLIMPORT to empty hides Windows export-annotation issues) for everyone who clones the repo. Recommend removing this file from the change.

Comment thread .clangd
ParameterNames: true
DeducedTypes: true
CompileFlags:
CompilationDatabase: build/ # Search build/ directory for compile_commands.json

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-ASCII smart quote / em-dash characters appear in the added comments (this line uses —; .gdbinit also uses — in its header). PostgreSQL sources and diffs must be ASCII only -- no smart quotes, em-dashes, or ellipsis characters. Replace with a plain hyphen if these files are kept at all (though the primary recommendation is to drop both files). [high confidence]


if [ "$DIVERGED" -gt 0 ]; then
# Check commit messages for "dev setup" or "dev v" pattern
DEV_SETUP_COMMITS=$(git log --format=%s upstream/master...origin/master | grep -iE "^dev (setup|v[0-9])" | wc -l)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Range operator inconsistency for DEV_SETUP_COMMITS. git log A...B (three-dot) computes the symmetric difference — commits reachable from either upstream/master or origin/master but not both. That includes upstream's new commits, so this count is inflated/misclassified and no longer reflects only local commits. To count local-only commits you want two-dot upstream/master..origin/master (as sync-upstream.yml line 61 correctly does). Since this count feeds the gating in the Attempt merge step that ultimately permits a --force-with-lease push, the mismatch between the two workflows is a real correctness hazard.

Suggested change
DEV_SETUP_COMMITS=$(git log --format=%s upstream/master...origin/master | grep -iE "^dev (setup|v[0-9])" | wc -l)
DEV_SETUP_COMMITS=$(git log --format=%s upstream/master..origin/master | grep -iE "^dev (setup|v[0-9])" | wc -l)

Comment thread flake.nix
inherit (shellConfig) gdbConfig flameGraphScript pgbenchScript;
};

environment.localBinInPath = true;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

environment.localBinInPath = true; is a NixOS module option, not a valid flake output. Here it is placed inside the per-system attribute set returned by flake-utils.lib.eachDefaultSystem, so it is dead: nothing consumes it and it has no effect. Remove it.

Comment thread shell.nix
ulimit -c unlimited

# Local git excludes
git config core.excludesFile .local-gitignore 2>/dev/null || true

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

commonHookTail rewrites the user's repo-local git config (core.excludesFile) as a side effect of merely entering the dev shell, and points it at .local-gitignore, which is not committed to the repository. This silently mutates the checkout's git configuration on every nix develop, and references a file that does not exist. Either commit .local-gitignore and make this opt-in, or drop the automatic git config mutation.

Comment thread shell.nix
-P 5 \
--log \
--log-prefix="$OUTPUT_DIR/pgbench_$TIMESTAMP" \
$BENCH_ARGS \

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$BENCH_ARGS is expanded unquoted under set -euo pipefail. It relies on word-splitting to pass multiple pgbench flags (e.g. -b select-only@70 -b tpcb-like@30), which works but is fragile and easy to break. Prefer a bash array (BENCH_ARGS=(); ...; "$PGBENCH" ... "${BENCH_ARGS[@]}" ...) so the intent is explicit and quoting-safe.

Comment thread pg-aliases.sh
if [ "$last_compiler" != "$current_compiler" ] && [ "$last_compiler" != "unknown" ]; then
echo "Detected compiler change from $last_compiler to $current_compiler"
echo "Cleaning build directory..."
trash "$build_dir" 2>/dev/null || rm -rf "$build_dir"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

trash is used as the primary destructive tool here (and in pg-full-clean, pg-init, and the valgrind wrapper cleanup) with an || rm -rf fallback, but trash/trash-cli is not among the packages declared in shell.nix getPostgreSQLDeps. Inside the dev shell trash will therefore never be found and the code always falls through to the irreversible rm -rf, contradicting the safety intent of using a trash bin. Either add trash-cli to the dependency list or drop the trash indirection and document that removal is permanent.

Comment thread pg-aliases.sh
ulimit -c unlimited
if ! [ -w /proc/sys/kernel/core_pattern ]; then
echo "Setting kernel.core_pattern (requires sudo)..."
echo "core.%p" | sudo tee /proc/sys/kernel/core_pattern >/dev/null || {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pg-enable-cores/pg-disable-cores mutate the system-wide sysctl /proc/sys/kernel/core_pattern via sudo tee. This is a host-global side effect from a dev-shell helper: it changes core-dump handling for every process on the machine, and pg-disable-cores unconditionally 'restores' the pattern to a hardcoded core/core.%p, which may not match the system's original value (silently clobbering a distro or systemd-coredump configuration). Combined with ulimit -c unlimited set on every shell entry, this is an easy-to-misuse footgun. Consider dropping the sysctl writes entirely (leave core_pattern to the operator) or at least save and restore the prior value instead of assuming a default.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 OCR found 17 issue(s).

  • 17 inline, 0 in summary


cmdline2 = psprintf("exec %s", cmdline);
execl(shellprog, shellprog, "-c", cmdline2, (char *) NULL);
execlp(shellprog, shellprog, "-c", cmdline2, (char *) NULL);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This execl -> execlp change is unrelated to the patch's purpose and should be dropped (minimal-diff discipline). shellprog is SHELLPROG, which is defined as $(SHELL) in the Makefiles and /bin/sh in meson.build -- always an absolute path. For an absolute path execlp behaves identically to execl, so this provides no functional benefit. Worse, execlp searches PATH when the program name contains no slash; if SHELLPROG were ever configured as a bare name, the test harness would resolve and exec a shell from a possibly attacker-influenced PATH -- a security regression in code that execs shell commands. Revert to execl. (high confidence)

Suggested change
execlp(shellprog, shellprog, "-c", cmdline2, (char *) NULL);
execl(shellprog, shellprog, "-c", cmdline2, (char *) NULL);

@@ -1,4 +1,4 @@
#!/usr/bin/perl
#!/usr/bin/env perl

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated shebang change; drop it to keep the diff minimal. Every other Perl script in the tree (e.g. src/tools/copyright.pl, src/tools/git_changelog, src/tools/mark_pgdllimport.pl) uses #!/usr/bin/perl. Switching only pgindent to #!/usr/bin/env perl makes it the sole outlier and does not belong in this patch -- this looks like a local dev-environment convenience (consistent with the added flake.nix/shell.nix). Revert to #!/usr/bin/perl. (high confidence)

Suggested change
#!/usr/bin/env perl
#!/usr/bin/perl

Comment thread .gdbinit
Comment on lines +19 to +20
# Main entry: heap_update
break heapam.c:3210

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This entire file does not belong in a commitfest-bound PR. Two independent, verifiable defects make it actively misleading:

  1. The hardcoded line-number breakpoints have already drifted. heapam.c:3210 is the closing brace of heap_delete() in the current tree, not heap_update() as this comment claims. Line-number breakpoints silently point at the wrong code the moment any surrounding line changes.

  2. Several function-name breakpoints below reference symbols that do not exist anywhere in the tree (heap_hot_indexed_create_tuple, heap_xlog_indexed_update, etc. -- see the comments on those lines). GDB will error on each unresolved breakpoint when the file is sourced.

Per the top-level .gitignore policy, local-workflow/editor scaffolding should be ignored via $GIT_DIR/info/exclude, not committed. Please drop this file from the patch.

Comment thread .gdbinit
Comment on lines +45 to +46
# Create augmented tuple with embedded modified-column bitmap
break heap_hot_indexed_create_tuple

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

heap_hot_indexed_create_tuple does not exist anywhere in the source tree (verified via search). The surrounding heap_hot_indexed_* breakpoints in this block are likewise unresolved symbols. Sourcing this .gdbinit will produce an error for each one. These names appear to come from an unmerged feature branch, so the file is non-functional as committed.

Comment thread .gdbinit
Comment on lines +144 to +145
# WAL replay for XLOG_HEAP2_INDEXED_UPDATE
break heap_xlog_indexed_update

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neither heap_xlog_indexed_update nor the WAL opcode XLOG_HEAP2_INDEXED_UPDATE exists in the tree (verified via search). This breakpoint will fail to resolve on load.

Comment thread src/include/storage/buf_internals.h Outdated
* max") reads naturally.
*/
#define BM_MAX_USAGE_COUNT 5
#define BM_MAX_USAGE_COUNT BUF_COOLSTATE_HOT

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Retaining the name BM_MAX_USAGE_COUNT for what is now simply BUF_COOLSTATE_HOT (value 1) is misleading: it is no longer a usage count nor the maximum of a counter, yet it is still used to size arrays in pg_buffercache (usage_counts[BM_MAX_USAGE_COUNT + 1]) and the loop that emits usage-count buckets. The StaticAssertDecl message was also reworded to "cooling state doesn't fit..." while the asserted macro still carries the old name, so the message and the symbol disagree. Prefer referencing BUF_COOLSTATE_HOT directly at the use sites rather than keeping a historical alias whose name no longer describes its value. Confidence: moderate.

forknum = BufTagGetForkNum(&bufHdr->tag);
blocknum = bufHdr->tag.blockNum;
usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
usagecount = BUF_STATE_GET_COOLSTATE(buf_state);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a user-visible contract change, not a mechanical rename. usagecount previously ranged over 0..5 and is documented in pgbuffercache.sgml as "Clock-sweep access count"; it now only ever reports 0 or 1. Monitoring queries, histograms, and dashboards built on the 0..5 domain will silently break. The same applies to pg_buffercache_summary.usagecount_avg (now bounded to [0,1]) and pg_buffercache_usage_counts (now only ever emits buckets 0 and 1). The pgbuffercache docs are not updated in this change and still describe the old semantics. A user-visible change like this needs the documentation updated to describe the new HOT/COOL domain. Confidence: high.

Comment thread src/backend/storage/buffer/bufmgr.c Outdated
Comment on lines +2337 to +2339
/* Admit the newly loaded page COOL (probation); a second access via
* PinBuffer promotes it to HOT. This is what makes a one-touch scan
* self-evicting -- see the cooling-state notes in buf_internals.h. */

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Multi-line block comment does not follow the PostgreSQL comment style: the opening /* must be on its own line with the text starting on the next line (see the comment immediately above this hunk). As written this will not survive pgindent cleanly. Confidence: high.

Suggested change
/* Admit the newly loaded page COOL (probation); a second access via
* PinBuffer promotes it to HOT. This is what makes a one-touch scan
* self-evicting -- see the cooling-state notes in buf_internals.h. */
/*
* Admit the newly loaded page COOL (probation); a second access via
* PinBuffer promotes it to HOT. This is what makes a one-touch scan
* self-evicting; see the cooling-state notes in buf_internals.h.
*/

Comment thread src/backend/storage/buffer/bufmgr.c Outdated
Comment on lines +3009 to +3010
/* Admit COOL (probation); see the comment at the other admission
* site and the cooling-state notes in buf_internals.h. */

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same block-comment style problem here: /* should be on its own line for a multi-line comment, per surrounding convention and pgindent. Confidence: high.

Suggested change
/* Admit COOL (probation); see the comment at the other admission
* site and the cooling-state notes in buf_internals.h. */
/*
* Admit COOL (probation); see the comment at the other admission
* site and the cooling-state notes in buf_internals.h.
*/

Comment thread src/backend/storage/buffer/bufmgr.c Outdated
Comment on lines +2336 to +2337
set_bits |= BM_TAG_VALID;
/* Admit the newly loaded page COOL (probation); a second access via

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Behavioral regression / policy change (high confidence): stock code admits a demand-loaded page with usage_count=1, giving it one full clock-sweep pass of grace before it becomes an eviction candidate. Admitting COOL (0) makes every newly read-in page immediately reclaimable on the very first sweep tick that reaches it, before any opportunity for a second access to promote it. This affects ALL workloads, not just one-touch scans: a page that is read in, used once, and needed again shortly after can now be evicted a tick later and re-read. This is a fundamental buffer-replacement policy change. Per pgsql-hackers norms it needs (a) a reproducible benchmark showing it does not regress non-scan workloads, (b) regression/TAP tests, and (c) a reference to the design thread. Without those this reads as WIP, not commit-ready.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 OCR found 20 issue(s).

  • 19 inline, 1 in summary

📄 .gitignore

[high confidence] This whole change set (.clangd, .gdbinit, flake.nix, shell.nix, pg-aliases.sh, and the glibc-no-fortify-warning.patch it pulls in) is personal developer-environment tooling. The repository's own policy, stated right here at the top of .gitignore, is that "Auxiliary files from local workflows, your preferred editor, etc. should be ignored locally using $GIT_DIR/info/exclude or ~/.gitexclude." None of these files belong in a patch destined for pgsql-hackers/commitfest; they add non-minimal, unrelated content and will draw immediate rejection. Keep them in a private $GIT_DIR/info/exclude instead. (This comment is anchored here only because .gitignore is the stable file documenting the policy; the files themselves should simply not be committed.)


cmdline2 = psprintf("exec %s", cmdline);
execl(shellprog, shellprog, "-c", cmdline2, (char *) NULL);
execlp(shellprog, shellprog, "-c", cmdline2, (char *) NULL);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This execl->execlp change is a functional no-op given the current build config, and is likely unwanted. shellprog comes from SHELLPROG, which is $(SHELL) (an absolute path like /bin/sh) in the make builds and the literal /bin/sh in the meson build. Per POSIX, execlp only performs a PATH search when the program name contains no slash; since shellprog always contains a slash here, execlp behaves identically to execl. So this does not change behavior for the shipped configuration, while introducing a footgun: if SHELLPROG were ever a bare name, the regression harness would start resolving the shell via $PATH, which is a less predictable/less secure choice for a test driver. (high confidence) Unless there is a specific need to allow a PATH-resolved shell (which should be justified with a -hackers reference), revert this to execl to keep the diff minimal.

Suggested change
execlp(shellprog, shellprog, "-c", cmdline2, (char *) NULL);
execl(shellprog, shellprog, "-c", cmdline2, (char *) NULL);

@@ -1,4 +1,4 @@
#!/usr/bin/perl
#!/usr/bin/env perl

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shebang change makes pgindent the only Perl script in the tree using #!/usr/bin/env perl; every other .pl/Perl tool (e.g. genbki.pl, gen_node_support.pl, copyright.pl, mark_pgdllimport.pl, and ~30 others) uses #!/usr/bin/perl. The change is inconsistent with the established project-wide convention and is unrelated to any functional fix here. (high confidence) Revert to keep the diff minimal and consistent; if switching to env perl is intended, it is a separate, tree-wide decision that belongs on -hackers, not a one-off edit to this file.

Suggested change
#!/usr/bin/env perl
#!/usr/bin/perl

Comment thread .clangd
Comment on lines +1 to +2
Diagnostics:
MissingIncludes: None

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[high confidence] This is personal developer-environment config and must not be committed to a patch destined for pgsql-hackers/commitfest. The repository's top-level .gitignore explicitly directs such files to a local $GIT_DIR/info/exclude or ~/.gitexclude. Beyond that policy, -I../../../../src/include and CompilationDatabase: build/ assume a fixed working-directory depth and build layout, so this is brittle and environment-specific. Drop this file from the patch.

Comment thread .gdbinit
Comment on lines +19 to +20
# Main entry: heap_update
break heapam.c:3210

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[high confidence] This developer-only debugging file must not be in a commitfest patch (see the local-workflow policy in .gitignore). Worse, it hardcodes line-number breakpoints such as heapam.c:3210, heapam.c:4019, and pruneheap.c:1802 that are tied to the exact feature diff. These silently rot the moment any of that code shifts by a line, so even as a dev aid they are a maintenance trap. Remove this file from the patch and keep it in a private local location.

Comment thread flake.nix
Comment on lines +1 to +2
{
description = "PostgreSQL development environment";

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[high confidence] The Nix flake is local developer-environment setup and does not belong in a pgsql-hackers patch (see the local-workflow policy in the repo's .gitignore). It also bundles unrelated concern(s) with the actual feature work in this PR (the buffer cooling-state changes in bufmgr.c). Remove this file from the patch.

Comment thread src/backend/storage/buffer/bufmgr.c Outdated
Comment on lines +3359 to +3361
if (strategy == NULL &&
BUF_STATE_GET_COOLSTATE(buf_state) < BUF_COOLSTATE_HOT)
buf_state += BUF_COOLSTATE_ONE;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This replaces the 0..5 clock-sweep usage_count with a single HOT/COOL bit, a fundamental change to the buffer replacement algorithm. A HOT buffer now survives exactly one sweep pass before becoming a victim, versus up to 5+ full clock cycles of retention before. This drastically shortens working-set retention and risks cache thrashing under mixed workloads. For a core replacement-algorithm change like this, pgsql-hackers will require: a reproducible benchmark demonstrating the claimed scan-resistance benefit and no regression on OLTP/working-set-heavy workloads, a reference to the design thread (Message-Id), and ideally the ability to compare against the existing algorithm. As submitted there is no benchmark, no GUC, and no regression test exercising the new eviction behavior. (high confidence)

Comment thread src/include/storage/buf_internals.h Outdated
* max") reads naturally.
*/
#define BM_MAX_USAGE_COUNT 5
#define BM_MAX_USAGE_COUNT BUF_COOLSTATE_HOT

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Retaining the name BM_MAX_USAGE_COUNT for a value that is now the max cooling state (1, not 5) is misleading, and the surrounding code mixes the old BUF_USAGECOUNT_* nomenclature (used to clear the cooling bit via &= ~BUF_USAGECOUNT_MASK) with the new BUF_COOLSTATE_* names. This dual vocabulary makes the subsystem harder to follow. Also note the comment at UnlockBufHdrExt() (around line 506) still reads "this approach would not trivially work for usagecount, since we need to cap the usagecount at BM_MAX_USAGE_COUNT" -- that reasoning is now stated in stale usagecount terms. Prefer renaming the field/macros consistently to the cooling-state vocabulary (or keep usagecount throughout); avoid aliasing one onto the other. If any out-of-tree extension reads BM_MAX_USAGE_COUNT, the silent redefinition from 5 to 1 is also a surprise. (moderate confidence)

Comment thread src/include/storage/buf_internals.h Outdated
* The field historically used for the 0..5 usage_count now holds a single
* cooling-state bit: HOT (recently accessed, not an eviction candidate) or
* COOL (an eviction candidate). We reuse BUF_USAGECOUNT_ONE as the unit so
* the buffer-state bit geography -- refcount, flag, and lock offsets, and the

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostgreSQL source requires ASCII-only comments and the house style is WHY-not-WHAT. These new comment blocks use "--" as em-dash-like separators in running prose ("bit geography -- refcount...", "a sequential scan -- therefore") and are long narrative descriptions of what the algorithm does. Tighten to concise WHY comments and drop the em-dash-style "--" usage. (low confidence)

Comment thread src/backend/storage/buffer/freelist.c Outdated
local_buf_state -= BUF_USAGECOUNT_ONE;
/*
* HOT buffer: cool it in place this tick and keep scanning. We
* do NOT claim it now -- a demoted buffer only becomes a victim

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same ASCII/comment-style note as buf_internals.h: these new comments use em-dash-like "--" in prose ("we\n * do NOT claim it now --", "HOT working set instead -- tell the") and describe WHAT at length. Keep comments ASCII and WHY-focused per the tree's conventions. (low confidence)

Comment thread flake.nix
inherit (shellConfig) gdbConfig flameGraphScript pgbenchScript;
};

environment.localBinInPath = true;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

environment.localBinInPath is a NixOS module option and has no meaning as a flake output. Inside flake-utils.lib.eachDefaultSystem it becomes <system>.environment.localBinInPath, which is not a recognized flake output schema attribute and does nothing. This looks copy-pasted from a configuration.nix; remove it. (This is in addition to the already-confirmed finding that the whole flake is dev-only tooling that doesn't belong in a commitfest patch.)

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 OCR found 28 issue(s).

  • 25 inline, 3 in summary (inline capped at 25)

📄 src/include/storage/buf_internals.h (L103-L107)

The comment's derivation does not match the arithmetic. PG_CACHE_LINE_SIZE / sizeof(uint32) (= 128/4 = 32) is the number of uint32s in a cache line, not "one cache line of buffer descriptors' worth of demotions" -- a BufferDesc is ~64 bytes, so a cache line holds far fewer than 32 of them, and the field this acts on is 64-bit buffer state, not a bare uint32. The stated hardware justification is spurious; either pick a value with a real rationale or drop the cache-line framing. Confidence: high.


📄 src/include/storage/buf_internals.h (L106-L107)

Design concern: this threshold is a fixed 32 regardless of NBuffers, yet the comment itself argues the starvation window scales with NBuffers (pass time T grows with the pool). On a large shared_buffers, a single allocation abandons the probation rule after cooling only 32 buffers out of potentially millions -- so under even brief, localized pressure the clock sweep starts claiming still-HOT buffers almost immediately, defeating the scan-resistance the whole cooling design is meant to provide. The giving-up point should scale with NBuffers (or be justified against it) rather than being a tiny hardware-derived constant. Needs a benchmark on a large pool to show HOT working-set pages are not prematurely evicted. Confidence: moderate.


📄 src/include/storage/bufmgr.h (L34-L37)

This new block comment is stacked directly on top of the pre-existing block comment for the same enum (the "Possible arguments for GetAccessStrategy()" comment just above), leaving two adjacent comment blocks describing one declaration. Beyond the awkward stacking, the content is a design-rationale and manual-testing essay ("To test whether vacuum's ring is still earning its keep, set vacuum_buffer_usage_limit = 0 ...") that explains why rings were kept and how to experiment with removing them -- that belongs in the commit message / -hackers thread, not in a header comment above a type definition. A header comment should say what BufferAccessStrategyType is. Fold any still-relevant note into the existing comment and move the rationale/testing prose out. Confidence: high.

forknum = BufTagGetForkNum(&bufHdr->tag);
blocknum = bufHdr->tag.blockNum;
usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
usagecount = BUF_STATE_GET_COOLSTATE(buf_state);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a silent, user-visible semantic change to the usage_count column of the pg_buffercache view. BUF_STATE_GET_COOLSTATE() returns only 0 (COOL) or 1 (HOT), whereas usage_count previously ranged 0..5. The column name and the local usagecount now misrepresent what is reported (a cool/hot bit, not a usage count) - a POLA violation for existing users and tooling that queries this column. The documented column description and example output in doc/src/sgml/pgbuffercache.sgml are now stale. Either rename the column/field to reflect the cool/hot state and update the docs, or document the new 0/1 semantics. Confidence: high.

{
buffers_used++;
usagecount_total += BUF_STATE_GET_USAGECOUNT(buf_state);
usagecount_total += BUF_STATE_GET_COOLSTATE(buf_state);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

usagecount_total now accumulates the cool/hot bit (0/1), so the derived usagecount_avg column of pg_buffercache_summary() becomes the fraction of HOT buffers (0.0-1.0) rather than the former average usage count (0..~5, e.g. the documented 3.141129 example). This is a silent behavior change under an unchanged column name. The accompanying docs/example in pgbuffercache.sgml are not updated. Rename/redefine the column or document the new meaning. Confidence: high.

Comment thread .gdbinit
Comment on lines +6 to +7
# These breakpoints cover the major code paths introduced or modified by
# the HOT indexed updates patch series. They are organized by subsystem

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This entire file targets a "HOT indexed updates" feature whose symbols do not exist anywhere in this tree. I verified that heap_hot_indexed_tuple_size, heap_hot_indexed_create_tuple, heap_hot_search_buffer's indexed-update variants, heap_xlog_indexed_update, and HEAP_INDEXED_UPDATED return zero matches under src/. The actual source change in this PR is buffer-cooling state (BUF_COOLED, BUF_COOLSTATE_HOT, StrategyCoolClaims) in bufmgr.c/freelist.c -- a completely different, unrelated feature. These breakpoints are orphaned scaffolding from a different patch series and would mislead or fail to resolve for anyone sourcing the file. This file should not be committed (and none of this developer-local tooling belongs in an upstream patch).

Comment thread .gdbinit
Comment on lines +27 to +29
break heapam.c:4019
break heapam.c:4024
break heapam.c:4033

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hardcoded line-number breakpoints are already wrong against this tree. For example, the comment claims heapam.c:4019 is the "pure HOT (no indexed columns changed)" decision, but line 4019 of heapam.c in this repo is the generic re-fit check if (newtupsize > pagefree || ...). Line-number breakpoints are inherently brittle -- any edit shifts them so they silently point at unrelated code. Prefer function-name breakpoints (as used elsewhere in this file) over file:line form.

Comment thread pg-aliases.sh
alias pg-full-clean='trash "$PG_BUILD_DIR" "$PG_INSTALL_DIR" 2>/dev/null || rm -rf "$PG_BUILD_DIR" "$PG_INSTALL_DIR"; echo "Build and install directories cleaned"'

# Database management
alias pg-init='trash "$PG_DATA_DIR" 2>/dev/null || rm -rf "$PG_DATA_DIR"; "$PG_INSTALL_DIR/bin/initdb" --debug --no-clean "$PG_DATA_DIR"'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

trash is the primary deletion command here but is never declared in shell.nix buildInputs (I confirmed it does not appear in shell.nix/flake.nix). On systems without trash, this falls through to an unconditional rm -rf "$PG_DATA_DIR". Since PG_DATA_DIR is set via the dev-shell hook (/tmp/test-db-$(basename $PWD)), sourcing these aliases outside the dev shell leaves it unset, making this rm -rf "" -- a data-loss footgun. Either add trash (trash-cli) to the dev-shell inputs, or guard the deletions with [ -n "$PG_DATA_DIR" ].

Comment thread pg-aliases.sh
Comment on lines +6 to +8
pg_clean_for_compiler() {
local current_compiler="$(basename $CC)"
local build_dir="${1:-$PG_BUILD_DIR}"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The compiler-detection regex [gc]cc matches gcc and ccc, but not a plain cc symlink, and the first alternative also never matches a bare clang++. More importantly, basename $CC is unquoted (word-splits/globs if $CC ever contains spaces), and grep -o ... | head -1 | xargs basename will pick whichever absolute compiler path appears first in compile_commands.json, which is not necessarily the C compiler (it may be a C++ entry). This can both miss a real compiler switch (leaving a stale, mismatched build) and spuriously trigger a full rm -rf of the build dir. Consider comparing against the $build_dir/.compiler_used marker you already write instead of grepping the compile DB.

Comment thread pg-aliases.sh
Comment on lines +313 to +314
ASAN_OPTIONS="halt_on_error=0:abort_on_error=1:detect_leaks=0:print_summary=1:print_stacktrace=1" \
UBSAN_OPTIONS="halt_on_error=1:abort_on_error=1:print_stacktrace=1:print_summary=1" \

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pg-asan-regress builds the backend with -fsanitize=address but does not set LD_PRELOAD/verify_asan_link_order or ASAN_OPTIONS=verify_asan_link_order=0, and more critically runs the suite without detect_leaks disabled only for ASan while leaving UBSan halt_on_error=1. The bigger practical problem: ASan under PostgreSQL commonly aborts on shmget/mmap interception and on the fork()-heavy postmaster unless ASAN_OPTIONS includes abort_on_error=1 together with handling for detect_leaks across child processes; with halt_on_error=0 for ASan but halt_on_error=1 for UBSan, a single UBSan report in a backend kills that backend mid-test and desynchronizes pg_regress, producing confusing diffs rather than a clean sanitizer report. Please align the two option sets (and document the known-needed ASan options for a postmaster).

Comment thread src/backend/storage/buffer/freelist.c Outdated
Comment on lines 342 to 346
pg_atomic_fetch_add_u64(&StrategyControl->numCoolClaims, 1);
local_buf_state += BUF_REFCOUNT_ONE;

if (pg_atomic_compare_exchange_u64(&buf->state, &old_buf_state,
local_buf_state))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The numCoolClaims increment happens before the CAS, but the CAS can fail (another backend changed buf->state concurrently). On failure the inner loop falls through to the bottom and retries line 267 with the refreshed old_buf_state; if the buffer is still HOT and we are still at/above the threshold, we increment numCoolClaims again. Every CAS retry over-counts. Since BgBufferSync() keys its pre-cooling decision purely on the delta cool_claims > prev_cool_claims, this inflates the signal and can trigger background cooling under contention even when no allocation actually failed to find a COOL victim. Move the increment to only fire on a successful claim (after the CAS succeeds). Confidence: high.

Suggested change
pg_atomic_fetch_add_u64(&StrategyControl->numCoolClaims, 1);
local_buf_state += BUF_REFCOUNT_ONE;
if (pg_atomic_compare_exchange_u64(&buf->state, &old_buf_state,
local_buf_state))
local_buf_state += BUF_REFCOUNT_ONE;
if (pg_atomic_compare_exchange_u64(&buf->state, &old_buf_state,
local_buf_state))

Comment thread src/backend/storage/buffer/bufmgr.c Outdated
Comment on lines 4237 to 4250
if (cool_if_hot &&
BUF_STATE_GET_REFCOUNT(buf_state) == 0 &&
BUF_STATE_GET_COOLSTATE(buf_state) != BUF_COOLSTATE_COOL)
{
UnlockBufHdrExt(bufHdr, buf_state, 0, BUF_USAGECOUNT_MASK, 0);
buf_state = LockBufHdr(bufHdr);
result |= BUF_COOLED;
}

if (BUF_STATE_GET_REFCOUNT(buf_state) == 0 &&
BUF_STATE_GET_USAGECOUNT(buf_state) == 0)
BUF_STATE_GET_COOLSTATE(buf_state) == BUF_COOLSTATE_COOL)
{
result |= BUF_REUSABLE;
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Double-count: when cool_if_hot demotes a buffer above (lines 4237-4244), buf_state is re-read via LockBufHdr and is now COOL with refcount 0, so this block also sets BUF_REUSABLE on the same buffer. Back in BgBufferSync() the buffer then increments reusable_buffers twice -- once for BUF_REUSABLE and once for BUF_COOLED. This compounds the already-flagged problem of counting a just-cooled buffer as reusable: it is counted twice, further inflating reusable_buffers and causing the LRU scan to stop prematurely. A freshly demoted buffer is not an eviction candidate yet (an intervening access re-promotes it), so it should not satisfy the BUF_REUSABLE test in the same call that cooled it. Confidence: high.

Comment on lines 849 to 850
return true;
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spurious trailing blank line added at end of file. git diff --check will flag this and it is unrelated whitespace churn; remove it. Confidence: high.

Suggested change
return true;
}
return true;
}

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 OCR found 19 issue(s).

  • 19 inline, 0 in summary

@@ -1,4 +1,4 @@
#!/usr/bin/perl
#!/usr/bin/env perl

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes pgindent the only Perl script in the tree not using the project's #!/usr/bin/perl convention; 30+ other .pl/Perl tools (genbki.pl, gen_node_support.pl, copyright.pl, etc.) still use the FHS absolute path. If the intent is non-FHS (e.g. Nix) portability, a single-file change doesn't achieve it while introducing an inconsistency. Either drop this hunk or convert the whole tree under a dedicated -hackers discussion. Confidence: moderate.

Suggested change
#!/usr/bin/env perl
#!/usr/bin/perl


cmdline2 = psprintf("exec %s", cmdline);
execl(shellprog, shellprog, "-c", cmdline2, (char *) NULL);
execlp(shellprog, shellprog, "-c", cmdline2, (char *) NULL);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switching to execlp has no effect for any in-tree build: SHELLPROG is $(SHELL) in the makefiles and /bin/sh in meson.build, all of which contain a slash. execlp only triggers a PATH search when the program name has no slash, so it behaves identically to execl here. This change only matters if SHELLPROG is a bare command name (e.g. sh), which is the Nix/non-FHS scenario implied by the other files in this changeset. As a standalone diff it is speculative scaffolding with no in-tree configuration exercising the new path. If the PATH-lookup behavior is actually intended/needed, the commit message should state why and ideally be accompanied by the build-config change that sets SHELLPROG to a bare name; otherwise this hunk is not minimal. (low confidence)

Comment thread .gdbinit
@@ -0,0 +1,156 @@
# HOT Indexed Updates — GDB breakpoints for code review

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This entire file is personal developer scaffolding and does not belong in a patch destined for pgsql-hackers/commitfest. It references a feature that is not present in this change: a codebase search confirms none of the heap_hot_indexed_* symbols (e.g. heap_hot_indexed_create_tuple, heap_hot_indexed_read_bitmap) nor XLOG_HEAP2_INDEXED_UPDATE / heap_xlog_indexed_update exist anywhere under src/. A .gdbinit tied to a nonexistent feature is dead weight that will draw immediate rejection on-list. This file (together with .clangd, pg-aliases.sh, flake.nix, shell.nix) should be kept out of the tree (e.g. via a personal/global gitignore), not committed.

Comment thread .gdbinit
# =========================================================================

# Main entry: heap_update
break heapam.c:3210

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These hardcoded source-line breakpoints are stale/wrong and actively misleading. Verified against the current tree: heapam.c:3210 is the closing brace of heap_delete (not the heap_update entry); heapam.c:4019 is a PageGetHeapFreeSpace check; heapam.c:4024 is inside a comment; heapam.c:4033 is a break;. None match the annotated HOT-indexed logic, and all line-number breakpoints silently drift after any edit to these files. If this file is kept at all, use function-name breakpoints (as the rest of the file already does) instead of file:line.

Comment thread pg-aliases.sh
# ============================================================
pg_clean_for_compiler() {
local current_compiler="$(basename $CC)"
local build_dir="${1:-$PG_BUILD_DIR}"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Destructive footgun: build_dir defaults to ${1:-$PG_BUILD_DIR}, and if PG_BUILD_DIR is unset/empty the subsequent trash "$build_dir" / rm -rf "$build_dir" operates on an empty string. Guard against an empty path before deleting, e.g. [ -n "$build_dir" ] || { echo "build_dir unset" >&2; return 1; }.

Suggested change
local build_dir="${1:-$PG_BUILD_DIR}"
local build_dir="${1:-$PG_BUILD_DIR}"
if [ -z "$build_dir" ]; then
echo "Error: build_dir is empty (PG_BUILD_DIR unset?)" >&2
return 1
fi

Comment on lines 658 to 659
usage_count = BUF_STATE_GET_COOLSTATE(buf_state);
usage_counts[usage_count]++;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

User-visible/documented behavior change with no doc or test update. These three functions previously reported usage_count over the documented 0..5 range; after switching to BUF_STATE_GET_COOLSTATE they report only 0 or 1. pg_buffercache_usage_counts() in particular changes the SQL-visible result set: the loop at usage_counts[usage_count]++ and the output loop are bounded by BM_MAX_USAGE_COUNT+1, which is now 2 instead of 6, so the view silently drops rows for counts 2..5. doc/src/sgml/pgbuffercache.sgml (describing usage_count) and the expected regression output are not updated. Per pgsql-hackers standards this is WIP: a documented view's semantics cannot change without updating the docs and regression tests, and ideally without a column-meaning rename to avoid a silent POLA violation.

Comment on lines 847 to 848
return true;
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stray trailing blank line added at end of file; git diff --check will flag this. Remove the extra blank line to keep the diff minimal.

Comment thread src/include/storage/buf_internals.h Outdated
Comment on lines +88 to +90
#define BUF_COOLSTATE_COOL 0
#define BUF_COOLSTATE_HOT 1
#define BUF_COOLSTATE_ONE BUF_USAGECOUNT_ONE

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a core buffer-replacement policy change (clock-sweep 0..5 usage_count replaced by a single-bit HOT/COOL 2Q-style scheme) on the buffer-manager hot path, plus a new background pre-cooling heuristic. It ships with no regression/TAP tests and no documentation, and the performance claims baked into these comments (self-evicting one-touch scans, reduced foreground starvation) are asserted without any reproducible benchmark or reference to a design discussion/-hackers thread. Per project standards this is WIP, not commit-ready: the behavior and performance change needs tests covering the eviction/scan-resistance edge paths, documentation, and a Message-Id reference for the design.

Comment thread src/include/storage/buf_internals.h Outdated
Comment on lines +103 to +107
* workloads never reach it. One cache line of buffer descriptors' worth of
* demotions is a cheap, hardware-derived choice.
*/
#define BUF_COOL_CLAIM_THRESHOLD \
(PG_CACHE_LINE_SIZE / sizeof(uint32))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Misleading unit in the rationale. BUF_COOL_CLAIM_THRESHOLD expands to PG_CACHE_LINE_SIZE / sizeof(uint32) = 128/4 = 32, i.e. one cache line's worth of uint32s, not of buffer descriptors. A BufferDesc is far larger than 4 bytes (and cache-line aligned), so "one cache line of buffer descriptors' worth of demotions" is wrong by ~an order of magnitude. Either correct the wording or derive the threshold from sizeof(BufferDesc) if descriptors were actually intended. (high confidence)

Comment thread src/include/storage/bufmgr.h Outdated
Comment on lines +34 to +37
/*
* Buffer access strategies.
*
* Note on the cooling-stage evictor: admitting a demand-loaded page COOL makes

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This adds a second comment block immediately after the existing enum comment (lines 28-33), leaving two stacked comments before the typedef. The new block is a mailing-list-style design essay (why the rings are kept, how to test removing vacuum's ring) rather than a description of what BufferAccessStrategyType is. Per PostgreSQL comment discipline, header comments should explain the declared type's contract, not carry the patch's design justification; move this rationale to the commit message / -hackers thread and keep the header comment minimal. (moderate confidence)

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 OCR found 26 issue(s).

  • 25 inline, 1 in summary (inline capped at 25)

📄 src/include/storage/buf_internals.h (L106-L107)

The rationale for this magic number is self-contradictory. PG_CACHE_LINE_SIZE / sizeof(uint32) evaluates to 32 (128/4), but the comment calls it "one cache line of buffer descriptors' worth of demotions." A BufferDesc is padded to BUFFERDESC_PAD_TO_SIZE (64 bytes on 64-bit), so a cache line holds at most 2 descriptors, not 32 -- and sizeof(uint32) is not the size of anything being demoted here. Either derive the threshold from a quantity it actually represents (and fix the comment), or justify 32 directly instead of dressing it up as a hardware-derived value.

Comment thread .gdbinit
Comment on lines +42 to +46
# Predict augmented tuple size (returns 0 if t_hoff would overflow)
break heap_hot_indexed_tuple_size

# Create augmented tuple with embedded modified-column bitmap
break heap_hot_indexed_create_tuple

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This .gdbinit (and the whole review group: .clangd, flake.nix, shell.nix, pg-aliases.sh) is personal developer-environment scaffolding unrelated to the actual change under review, which is a buffer-manager cooling-state rework (bufmgr.c SyncOneBuffer/cool_if_hot, buf_internals.h BUF_COOLSTATE_*, freelist.c StrategyCoolClaims). For a patch destined for pgsql-hackers/commitfest these files violate the minimal-diff rule and will draw immediate rejection; they should not be committed.

Worse, the breakpoints here target a completely different, nonexistent feature: the heap_hot_indexed_* functions (heap_hot_indexed_tuple_size, heap_hot_indexed_create_tuple, heap_hot_indexed_serialize_bitmap, etc.) do not exist anywhere in src/backend/access/heap/*.c in this tree. GDB will reject these as unresolved symbols. This confirms the file is stale scaffolding copied from an unrelated "HOT indexed updates" patch series and bears no relation to the buffer-manager change actually being reviewed. Remove the file from the patch.

Comment thread .gdbinit
Comment on lines +19 to +20
# Main entry: heap_update
break heapam.c:3210

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hardcoded absolute source line numbers (heapam.c:3210, heapam.c:4019, heapam.c:4033, heapam.c:4101, pruneheap.c:1802, execIndexing.c:370, etc.) will silently drift and attach to the wrong lines as the source evolves. Line-number breakpoints are inherently unmaintainable in a shared repo; combined with the stale heap_hot_indexed_* symbol references, this file is misleading. (Low severity because it is tooling, not shipped code -- the primary issue is that the file should not be in the patch at all.)

Comment thread flake.nix
inherit (shellConfig) gdbConfig flameGraphScript pgbenchScript;
};

environment.localBinInPath = true;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

environment.localBinInPath is a NixOS module option, not a valid per-system flake output. Placing it alongside formatter/devShells/packages in the eachDefaultSystem result set is meaningless here -- nothing consumes it (it is referenced only at this line), and it does not configure PATH for the dev shells as presumably intended. It is dead/misplaced output and should be removed (or moved into an actual NixOS module if that behavior is wanted).

Comment thread pg-aliases.sh
local current_compiler="$(basename $CC)"
local build_dir="${1:-$PG_BUILD_DIR}"

if [ -f "$build_dir/compile_commands.json" ]; then

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pg_clean_for_compiler takes build_dir="${1:-$PG_BUILD_DIR}" and later runs rm -rf "$build_dir" (line 16). If both the argument and $PG_BUILD_DIR are empty/unset, build_dir is empty and the trash/rm -rf "" fallback is a footgun in a destructive-command path. The same unguarded pattern applies to pg-full-clean (rm -rf "$PG_BUILD_DIR" "$PG_INSTALL_DIR") and pg-init (rm -rf "$PG_DATA_DIR"): an unset variable makes these operate on nothing or the wrong path. Guard against empty/unset variables (e.g. [ -n "$build_dir" ] || return 1) before any destructive deletion. (The deeper issue remains that this helper script should not be part of a pgsql-hackers patch.)

Comment thread pg-aliases.sh
ulimit -c unlimited
if ! [ -w /proc/sys/kernel/core_pattern ]; then
echo "Setting kernel.core_pattern (requires sudo)..."
echo "core.%p" | sudo tee /proc/sys/kernel/core_pattern >/dev/null || {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pg-enable-cores/pg-disable-cores mutate the system-wide kernel.core_pattern sysctl via sudo tee, a global side effect triggered from a shell helper. Together with the developer-specific absolute/temp paths embedded elsewhere (/tmp/test-db-$(basename $PWD), ~/.gdb_history_postgres, $HOME/.ccache/pg/...), this is inappropriate for a shared repository. (Low severity as it is opt-in tooling, but it reinforces that these files do not belong in the patch.)

Comment thread src/backend/storage/buffer/bufmgr.c Outdated
Comment on lines +3908 to +3909
/* Cool-claim count as of our previous cycle, to get a rate not a total */
static uint64 prev_cool_claims = 0;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prev_cool_claims is a per-process function-level static while numCoolClaims is a cluster-wide shared counter that is only zeroed at StrategyCtlShmemInit(). After a bgwriter crash/restart the static resets to 0 but the shared counter persists at a large value, so the first post-restart cycle sees cool_claims > prev_cool_claims and enables pre-cooling regardless of actual pressure. Self-correcting after one cycle, but worth a comment or seeding prev from the shared value on entry. [moderate confidence]

Comment on lines 847 to 848
return true;
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trailing blank line added at EOF. This is whitespace churn unrelated to the change and git diff --check / pgindent will flag it. Remove it. [high confidence]

Suggested change
return true;
}
return true;
}

Comment thread src/include/storage/bufmgr.h Outdated
Comment on lines +34 to +37
/*
* Buffer access strategies.
*
* Note on the cooling-stage evictor: admitting a demand-loaded page COOL makes

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two problems with this comment block. (1) It is a second, adjacent block comment immediately following the existing one at lines 28-33 on the same enum -- merge them rather than stacking two headers. (2) The content is design-discussion/aspirational narrative ("Measurement is the only way to settle whether their rings still earn their keep", suggested future experiments with vacuum_buffer_usage_limit) that describes work not done in this patch. Per project convention, comments describe what the code does now and explain WHY; this rationale belongs in the commit message / buffer README, not an enum header. [moderate confidence]

Comment thread src/include/storage/buf_internals.h Outdated
* max") reads naturally.
*/
#define BM_MAX_USAGE_COUNT 5
#define BM_MAX_USAGE_COUNT BUF_COOLSTATE_HOT

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This redefines the public symbol BM_MAX_USAGE_COUNT from 5 to BUF_COOLSTATE_HOT (=1) while keeping the name. That collapses the user-visible usage_count field (exposed via pg_buffercache_pages / pg_buffercache_usage_counts / pg_buffercache_summary) from the documented 0..5 range to 0/1. This is a user-visible behavior change: the pgbuffercache docs (doc/src/sgml/pgbuffercache.sgml example output still shows usage_count rows 0..5) and any regression output encoding usage_count semantics become stale, and no doc/test update accompanies this header change. A fundamental buffer-replacement-algorithm change like this needs tests and documentation to be commit-ready. [high confidence]

Comment thread src/backend/storage/buffer/freelist.c Outdated
Comment on lines +340 to +346
pg_atomic_fetch_add_u64(&StrategyControl->numCoolClaims, 1);
local_buf_state += BUF_REFCOUNT_ONE;

if (pg_atomic_compare_exchange_u64(&buf->state, &old_buf_state,
local_buf_state))
{
trycounter = NBuffers;
break;
/* Found a usable buffer */

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

numCoolClaims is incremented before the CAS. If the CAS fails (a concurrent backend pinned, cooled, or re-promoted the buffer), the counter was already bumped even though no HOT buffer was actually claimed, and the loop retries -- potentially bumping it again. Under header-state contention this over-counts the very signal BgBufferSync() reads via StrategyCoolClaims() to set cool_if_hot, biasing the background writer toward aggressive pre-cooling when real claim pressure is lower. Move the increment inside the successful-CAS branch so it counts actual claims only.

Suggested change
pg_atomic_fetch_add_u64(&StrategyControl->numCoolClaims, 1);
local_buf_state += BUF_REFCOUNT_ONE;
if (pg_atomic_compare_exchange_u64(&buf->state, &old_buf_state,
local_buf_state))
{
trycounter = NBuffers;
break;
/* Found a usable buffer */
local_buf_state += BUF_REFCOUNT_ONE;
if (pg_atomic_compare_exchange_u64(&buf->state, &old_buf_state,
local_buf_state))
{
pg_atomic_fetch_add_u64(&StrategyControl->numCoolClaims, 1);
/* Found a usable buffer */

ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 OCR found 24 issue(s).

  • 24 inline, 0 in summary

Comment thread .gdbinit
Comment on lines +27 to +29
break heapam.c:4019
break heapam.c:4024
break heapam.c:4033

ghost Oct 4, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These GDB breakpoints target a "HOT indexed updates" patch that is NOT present in this tree. Verified: heap_hot_indexed_create_tuple, heap_hot_indexed_tuple_size, heap_hot_indexed_serialize_bitmap, and heap_xlog_indexed_update do not exist anywhere in the source, so these function breakpoints will fail to resolve. The hardcoded line-number breakpoints are also wrong against the current tree: heapam_indexscan.c:182 is a PageGetItemId() call (not "redirect-with-data bitmap accumulator"), heapam_indexscan.c:250 is skip = false; (not "accumulate bitmap from INDEXED_UPDATED tuple"), and heapam_indexscan.c:297 is inside an unrelated static callback (not "stale entry detection"). This file is local developer scaffolding tightly coupled to one working-tree state and must not be committed to a patch destined for pgsql-hackers. (high confidence)

Comment thread flake.nix
inherit (shellConfig) gdbConfig flameGraphScript pgbenchScript;
};

environment.localBinInPath = true;

ghost Oct 4, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

environment.localBinInPath = true; is a NixOS module option, not a flake output. Placed here among the per-system eachDefaultSystem outputs it is meaningless and silently ignored; it does not enable anything. Remove it.

Comment on lines +14 to +15
issue_comment:
types: [created]

ghost Oct 4, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue_comment trigger fires on any comment to a PR, and pull_request fires on synchronize/reopened from any contributor's branch. The reusable workflow is then invoked with the OIDC-federated AWS role (aws_role_arn) and the repo GITHUB_TOKEN. There is no actor/author-association gate (e.g. a check that github.event.comment.user or the PR author is OWNER/MEMBER/COLLABORATOR). An unprivileged user can therefore drive Bedrock spend at will, and — if gburd/ci-workflows/.../ocr-review.yml checks out PR head code before using the AWS role — this is a classic poisoned-pipeline-execution (PPE) path leaking a cloud credential to untrusted code. Add an author_association/actor restriction to the if: for the comment path (and ideally for forked-PR events), and confirm the called workflow does not expose the role to attacker-controlled checkout. (high confidence on the missing gate; moderate on exploitability since the called workflow is not in this repo to inspect.)

(startsWith(github.event.comment.body, '/open-code-review') ||
startsWith(github.event.comment.body, '@open-code-review') ||
startsWith(github.event.comment.body, '/pg-history')))
uses: gburd/ci-workflows/.github/workflows/ocr-review.yml@v1

ghost Oct 4, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reusable workflow is pinned to the mutable tag @v1. Whoever controls gburd/ci-workflows (or an attacker who compromises it) can re-point v1 to arbitrary code that then runs with this repo's OIDC-federated AWS role and GITHUB_TOKEN. Pin to a full commit SHA (optionally with a # v1 trailing comment) so the referenced code is immutable.

Suggested change
uses: gburd/ci-workflows/.github/workflows/ocr-review.yml@v1
uses: gburd/ci-workflows/.github/workflows/ocr-review.yml@<full-commit-sha> # v1

# Needs repo variables AWS_ROLE_ARN, AWS_REGION, OCR_BEDROCK_MODEL. Auth is
# GitHub OIDC; no static AWS keys. See the ci-workflows README.

name: OCR AI Review

ghost Oct 4, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No permissions: block is declared at workflow or job level, so the default GITHUB_TOKEN scope (which depends on repo settings and can be broad) is passed to the called workflow. An AI review that posts PR comments needs at most pull-requests: write and contents: read. Declare least-privilege permissions explicitly rather than relying on repo defaults.

Suggested change
name: OCR AI Review
name: OCR AI Review
permissions:
contents: read
pull-requests: write


cmdline2 = psprintf("exec %s", cmdline);
execl(shellprog, shellprog, "-c", cmdline2, (char *) NULL);
execlp(shellprog, shellprog, "-c", cmdline2, (char *) NULL);

ghost Oct 4, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated change and a behavioral regression. shellprog comes from SHELLPROG, which the build sets to $(SHELL) (make) or the hardcoded absolute path /bin/sh (meson/src/test/regress/meson.build). The original execl runs exactly that path. execlp adds a $PATH search whenever the string contains no slash, so the test harness would resolve its shell via $PATH instead of the build-fixed path -- an unnecessary footgun (PATH injection / picking up an unintended shell) in a security-sensitive exec path.

This hunk is also unrelated to the buffer-manager work this PR centers on; per minimal-diff discipline it should not be bundled here. Unless there is a documented reason the regress runner must PATH-search for its shell, keep execl. (confidence: high)

Suggested change
execlp(shellprog, shellprog, "-c", cmdline2, (char *) NULL);
execl(shellprog, shellprog, "-c", cmdline2, (char *) NULL);

@@ -1,4 +1,4 @@
#!/usr/bin/perl
#!/usr/bin/env perl

ghost Oct 4, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated change that breaks tree-wide consistency. Every other Perl script in the tree (30+ files, e.g. genbki.pl, gen_node_support.pl, copyright.pl, mark_pgdllimport.pl) uses #!/usr/bin/perl; this edit makes pgindent the sole outlier using env perl. It is also unrelated to the buffer-manager changes this PR group concerns, so it is diff churn that will draw rejection on -hackers. Revert to the tree convention. (confidence: high)

Suggested change
#!/usr/bin/env perl
#!/usr/bin/perl

Comment thread src/backend/storage/buffer/freelist.c Outdated
Comment on lines +340 to +346
pg_atomic_fetch_add_u64(&StrategyControl->numCoolClaims, 1);
local_buf_state += BUF_REFCOUNT_ONE;

if (pg_atomic_compare_exchange_u64(&buf->state, &old_buf_state,
local_buf_state))
{
trycounter = NBuffers;
break;
/* Found a usable buffer */

ghost Oct 4, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

numCoolClaims is incremented before the CAS that actually claims the buffer. On CAS failure the inner for (;;) loop reloads old_buf_state and retries, so a single buffer under contention can bump the counter multiple times; worse, if the retry observes the buffer is now pinned (refcount != 0) we break out via the refcount branch without claiming anything, yet the counter was already incremented. This inflates the cool-claim rate that BgBufferSync() reads via StrategyCoolClaims() to set cool_if_hot, biasing the background writer toward pool-wide cooling under pressure that did not actually occur. Increment the counter only inside the successful-CAS branch, after the claim is committed.

(high confidence)

Suggested change
pg_atomic_fetch_add_u64(&StrategyControl->numCoolClaims, 1);
local_buf_state += BUF_REFCOUNT_ONE;
if (pg_atomic_compare_exchange_u64(&buf->state, &old_buf_state,
local_buf_state))
{
trycounter = NBuffers;
break;
/* Found a usable buffer */
local_buf_state += BUF_REFCOUNT_ONE;
if (pg_atomic_compare_exchange_u64(&buf->state, &old_buf_state,
local_buf_state))
{
pg_atomic_fetch_add_u64(&StrategyControl->numCoolClaims, 1);
/* Found a usable buffer */

Comment on lines 847 to 848
return true;
}

ghost Oct 4, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spurious trailing blank line added at end of file. git diff --check / pgindent will flag this as a whitespace defect; remove the extra blank line.

(high confidence)

Comment thread src/backend/storage/buffer/bufmgr.c Outdated
Comment on lines +4237 to +4244
if (cool_if_hot &&
BUF_STATE_GET_REFCOUNT(buf_state) == 0 &&
BUF_STATE_GET_COOLSTATE(buf_state) != BUF_COOLSTATE_COOL)
{
UnlockBufHdrExt(bufHdr, buf_state, 0, BUF_USAGECOUNT_MASK, 0);
buf_state = LockBufHdr(bufHdr);
result |= BUF_COOLED;
}

ghost Oct 4, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code already holds the header spinlock (from LockBufHdr above), yet to demote the buffer it calls UnlockBufHdrExt() -- which runs its own CAS retry loop and releases the lock -- and then re-acquires it with LockBufHdr(). That is two atomic operations plus a lock round-trip, directly contradicting the comment here ("this costs one masked store and no atomic retry") and the comment in BgBufferSync ("staging costs a masked store") and the function header ("a plain masked store rather than a CAS"). Since the spinlock is held, the bits can be cleared with a plain masked store on the already-read buf_state while keeping the lock, avoiding the unlock/relock entirely. As written, either the comments are wrong or the implementation is doing unnecessary work on the cleaning scan's path.

(high confidence)

ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 OCR found 35 issue(s).

  • 25 inline, 10 in summary (inline capped at 25)

📄 src/test/regress/pg_regress.c (L1246-L1246)

This execl -> execlp switch is an unrelated change bundled into a patch whose stated purpose is the buffer-manager work. On -hackers this is a top rejection reason ("patch does more than one thing"); it should be split into its own commit with a clear justification, or dropped.

On the substance: execlp() only differs from execl() when shellprog contains no slash, in which case it performs a PATH search. In the normal build SHELLPROG is /bin/sh (meson) or $(SHELL) (make, defaults to /bin/sh), both of which contain a slash, so behavior is identical. The only effect is to allow a bare program name in SHELLPROG to be resolved via PATH. If that is actually the goal, say so and update the surrounding comment (lines 1239-1242 explain why the shell is exec'd directly but nothing about PATH lookup). Note this also introduces a mild footgun: an untrusted PATH could cause a different sh to be executed than the one the build selected. (moderate confidence)


📄 contrib/pg_buffercache/pg_buffercache_pages.c (L164-L164)

Stale identifier after the mechanical substitution. The local is still named usagecount (and its column type is int2), but it now receives a HOT/COOL cooling-state bit (0 or 1), not a 0..5 usage count. The name actively misleads future readers into assuming the historical count semantics. Either keep the SQL column reporting a real count, or rename the variable to reflect that it now carries the cooling state so the code reads as if written that way. (moderate confidence)


📄 src/tools/pgindent/pgindent (L1-L1)

This change breaks the tree-wide shebang convention. Every other Perl script in the PostgreSQL source (genbki.pl, gen_node_support.pl, copyright.pl, mark_pgdllimport.pl, and ~30 more) uses #!/usr/bin/perl. Making pgindent the sole file to use #!/usr/bin/env perl is inconsistent, unrelated to any functional change, and would be rejected on -hackers as gratuitous churn. The in-tree ./configure/build process also rewrites the Perl path, so env provides no portability benefit here. Revert to match the established convention.

💡 Suggested change

Before:

#!/usr/bin/env perl

After:

#!/usr/bin/perl

📄 src/backend/storage/buffer/freelist.c (L340-L346)

numCoolClaims is incremented before the CAS, so it is over-counted on CAS failure. When the compare_exchange at the next line fails (concurrent modification), the inner for(;;) loop retries this same buffer; if it is still unpinned and HOT with cooled >= BUF_COOL_CLAIM_THRESHOLD, control reaches this fetch_add again and increments the counter a second (or Nth) time for a single eventual claim. StrategyCoolClaims() is read by BgBufferSync() purely as a rate signal (cool_claims > prev_cool_claims), so these spurious increments trigger spurious full-pool pre-cooling. Move the increment to the CAS-success branch, as is done for the pin in both the COOL branch and stock code. (high confidence)

💡 Suggested change

Before:

				pg_atomic_fetch_add_u64(&StrategyControl->numCoolClaims, 1);
				local_buf_state += BUF_REFCOUNT_ONE;

				if (pg_atomic_compare_exchange_u64(&buf->state, &old_buf_state,
												   local_buf_state))
				{
					/* Found a usable buffer */

After:

				local_buf_state += BUF_REFCOUNT_ONE;

				if (pg_atomic_compare_exchange_u64(&buf->state, &old_buf_state,
												   local_buf_state))
				{
					pg_atomic_fetch_add_u64(&StrategyControl->numCoolClaims, 1);
					/* Found a usable buffer */

📄 src/backend/storage/buffer/freelist.c (L847-L848)

Stray trailing blank line added at EOF; git diff --check / pgindent will flag this whitespace defect. Remove it. (high confidence)

💡 Suggested change

Before:

	return true;
}
+

After:

	return true;
}

📄 src/backend/storage/buffer/bufmgr.c (L4108-L4110)

prev_cool_claims is a function-local static uint64 initialized to 0, but StrategyCoolClaims() reads a process-wide shared atomic that may already be large (and never resets). On a bgwriter (re)start, prev_cool_claims == 0 while cool_claims is large, so cool_claims > prev_cool_claims is true on the first post-restart cycle, triggering a spurious cool-the-whole-pool pass even under no pressure. The existing code guards its first-cycle rate math with saved_info_valid; this new rate signal has no equivalent guard. At minimum seed prev_cool_claims from the current reading on the first cycle (e.g. gate it under saved_info_valid like the other prev_* state). (moderate confidence)


📄 src/backend/storage/buffer/bufmgr.c (L4241-L4243)

Unnecessary header-lock drop and re-acquire. UnlockBufHdrExt() runs its own CAS loop to clear the usagecount and release BM_LOCKED, then LockBufHdr() re-acquires the spinlock immediately. The buffer is already locked here; prefer clearing the cooling bits in-place (buf_state &= ~BUF_USAGECOUNT_MASK; pg_atomic_write to the still-locked state, or fold it into the existing UnlockBufHdr at function end) to avoid an extra atomic CAS plus a fresh spinlock acquire on this bgwriter path. Functionally it is correct -- the BUF_REUSABLE test below re-reads the freshly re-locked buf_state, so a concurrent PinBuffer() that promotes/pins between unlock and relock is handled -- but the drop/relock is wasteful. (moderate confidence)


📄 src/include/storage/buf_internals.h (L106-L107)

Comment is misleading about the derivation. BUF_COOL_CLAIM_THRESHOLD = PG_CACHE_LINE_SIZE / sizeof(uint32) = 32, but the comment calls this "one cache line of buffer descriptors' worth of demotions." A BufferDesc is far larger than a uint32, so one cache line does not hold 32 descriptors; the divisor is the historical state-field width, not sizeof(BufferDesc). Either fix the rationale text or pick a value whose justification matches the arithmetic. (low confidence)


📄 src/include/storage/buf_internals.h (L194-L194)

This replaces the 0..5 clock-sweep usage_count with a single HOT/COOL bit, which is a user-visible change: pg_buffercache's usagecount column and pg_buffercache_usage_counts() now only ever report 0 or 1 (never 2..5), and usagecount_avg collapses to a 0..1 range. No documentation update or regression/TAP test accompanies this replacement-policy change in the patch, and the scan-resistance / pre-cooling performance claims in the comments have no referenced reproducible benchmark or -hackers design thread. Per project standards this is WIP-level: add docs for the changed semantics and tests covering the new eviction/promotion/claim paths. (moderate confidence)


📄 src/backend/storage/buffer/bufmgr.c (L4143-L4144)

Double-counting of reusable_buffers for demoted buffers. A buffer that SyncOneBuffer() demotes via cool_if_hot returns BUF_COOLED, but after demotion it is unpinned and COOL, so SyncOneBuffer() also sets BUF_REUSABLE (bufmgr.c:4246-4249), or BUF_WRITTEN if it was dirty. Because this if (sync_state & BUF_COOLED) is a separate branch rather than part of the existing else if chain, every cooled buffer increments reusable_buffers twice: once at line 4127/4134 and again here. The over-count makes the LRU scan hit reusable_buffers < upcoming_alloc_est early and stop before it has staged enough candidates, and it also corrupts new_recent_alloc = reusable_buffers - reusable_buffers_est (line 4167), skewing the self-tuning smoothed_density for subsequent cycles. A BUF_COOLED buffer is already counted as reusable, so this extra increment should be removed (or the three outcomes made mutually exclusive). (high confidence)

Comment thread .gdbinit
Comment on lines +19 to +20
# Main entry: heap_update
break heapam.c:3210

ghost Oct 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These breakpoints target a "HOT indexed updates" patch series that is not present in this repository. Verified against the tree: the symbols heap_hot_indexed_*, HEAP_INDEXED_UPDATED, and heap_xlog_indexed_update do not exist (no matches), and the hardcoded line numbers do not match the referenced files. For example heapam_indexscan.c is only 588 lines, so heapam_indexscan.c:182 is lp = PageGetItemId(page, offnum);, :250 is a blank line, and :297 is a return in heapam_index_plain_tuple_getnext_slot() -- none of which match the comments ("initialize bitmap accumulator", "accumulate bitmap from INDEXED_UPDATED tuple", "stale entry detection"). As written, every symbol breakpoint fails to resolve and every line-number breakpoint lands on unrelated code, making this file misleading and non-functional for anyone debugging this tree.

Comment thread .gdbinit
@@ -0,0 +1,156 @@
# HOT Indexed Updates — GDB breakpoints for code review

ghost Oct 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GDB auto-sources ./.gdbinit from the working directory (subject to auto-load safe-path). A committed .gdbinit that silently installs dozens of breakpoints -- most at hardcoded line numbers that don't match this tree -- is a footgun: a contributor debugging an unrelated issue will have execution halt at unexpected places or see breakpoint-resolution errors on startup. This is developer-local tooling that does not belong in a committable PostgreSQL patch; it should be kept out of the tree (e.g. via $GIT_DIR/info/exclude as the project's own .gitignore header recommends) rather than committed.

Comment thread flake.nix
inherit (shellConfig) gdbConfig flameGraphScript pgbenchScript;
};

environment.localBinInPath = true;

ghost Oct 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

environment.localBinInPath is a NixOS module option, not a flake output. Placed here it becomes a per-system flake output attribute (environment.localBinInPath = true) produced by eachDefaultSystem, which nothing consumes -- flakes do not read environment.*. It is dead/misplaced config that only clutters the output schema; remove it.

Comment on lines +28 to +29
jobs:
review:

ghost Oct 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This caller declares no permissions: block, so the job inherits the repository's default GITHUB_TOKEN permissions (potentially broad). The repo's own convention (pg-ci.yml) explicitly restricts to contents: read. For a reusable-workflow caller, permissions must be set at the caller (workflow or job) level and are passed to the called workflow, which can only further restrict them. An AI review job that posts comments typically needs pull-requests: write (and contents: read); declare the least-privilege set explicitly here rather than relying on repo defaults.

Suggested change
jobs:
review:
permissions:
contents: read
pull-requests: write
jobs:
review:

Comment on lines +12 to +13
jobs:
check:

ghost Oct 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No permissions: block is declared, so this inherits the repository default GITHUB_TOKEN permissions. The comment says the job opens/updates an issue, which requires issues: write. Declare least-privilege permissions explicitly (consistent with pg-ci.yml's contents: read) instead of relying on repo defaults.

Suggested change
jobs:
check:
permissions:
contents: read
issues: write
jobs:
check:

Comment thread pg-aliases.sh
Comment on lines +100 to +107
alias pg-debug='
if command -v gdb >/dev/null 2>&1; then
pg-debug-gdb
elif command -v lldb >/dev/null 2>&1; then
pg-debug-lldb
else
echo "No debugger available (gdb or lldb required)"
fi'

ghost Oct 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These dispatcher aliases invoke other aliases (pg-debug-gdb / pg-debug-lldb, and below pg-attach-gdb / pg-attach-lldb) from inside an alias body. Aliases are only expanded on the first word of each simple command and alias expansion is not recursive within another alias's expansded body, so when pg-debug runs, the names pg-debug-gdb/pg-debug-lldb are treated as plain commands and will produce 'command not found' rather than launching the debugger. Define these as shell functions (which are resolved at call time) instead of aliases if the dispatch is intended to work.

Comment on lines +14 to +16
uses: gburd/ci-workflows/.github/workflows/ocr-model-check.yml@v1
with:
aws_role_arn: ${{ vars.AWS_ROLE_ARN }}

ghost Oct 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like the review workflow, this caller passes an aws_role_arn for a Bedrock check, which per the fork's design authenticates via GitHub OIDC. OIDC role assumption requires id-token: write on the caller's GITHUB_TOKEN, and a reusable workflow cannot grant itself that scope. With no permissions: block here, the inherited default may lack id-token: write, breaking auth. The separately-raised least-privilege finding also notes issues: write is needed to open/update the notification issue - so the required permissions are id-token: write + issues: write, not merely contents: read. (moderate confidence)


jobs:
check:
uses: gburd/ci-workflows/.github/workflows/ocr-model-check.yml@v1

ghost Oct 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reusable workflow is pinned to the mutable tag @v1. The upstream owner (or anyone who compromises that repo) can repoint v1 to any commit, and this job assumes an AWS OIDC role - repointed logic would run with your AWS credentials. Pin to a full commit SHA (e.g. ...@<40-char-sha> # v1) for third-party reusable workflows. (high confidence)

Comment thread shell.nix
Comment on lines +633 to +634
# Python UV
UV_PYTHON_DOWNLOADS=never

ghost Oct 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UV_PYTHON_DOWNLOADS=never is missing export, unlike every other variable in this shellHook. uv reads UV_PYTHON_DOWNLOADS from the process environment, so a non-exported shell variable has no effect and uv will still attempt managed Python downloads -- the intended isolation is silently a no-op. (moderate confidence)

Suggested change
# Python UV
UV_PYTHON_DOWNLOADS=never
# Python UV
export UV_PYTHON_DOWNLOADS=never

forknum = BufTagGetForkNum(&bufHdr->tag);
blocknum = bufHdr->tag.blockNum;
usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
usagecount = BUF_STATE_GET_COOLSTATE(buf_state);

ghost Oct 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Backward-compatibility / POLA concern (high confidence). This mechanical substitution silently changes the user-visible contract: the usage_count column of pg_buffercache (and pg_buffercache_usage_counts(), plus usagecount_avg of pg_buffercache_summary()) historically ranged 0..5 under clock-sweep. BUF_STATE_GET_COOLSTATE() now returns only 0 (COOL) or 1 (HOT), so these SQL functions start reporting a two-value cooling state under an unchanged column name. The variable here is still named usagecount and the output column is still usage_count, so a consumer parsing these views gets a different meaning with no signal. This needs to be surfaced to the author: either (a) keep exposing a compatible value, or (b) rename the user-facing columns and bump the pg_buffercache extension version, and in all cases update doc/src/sgml/pgbuffercache.sgml, which still documents the 0..5 range and shows a 0..5 example output. As-is the docs and the behavior now disagree.

ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 OCR found 36 issue(s).

  • 25 inline, 11 in summary (inline capped at 25)

📄 pg-aliases.sh

The heredoc expands $PG_SOURCE_DIR, $PG_BENCH_DIR, and $bindir into the generated wrapper unquoted. Note PG_SOURCE_DIR/pgSourceDir is $PWD and PG_DATA_DIR/benchmark dirs are derived from $(basename $PWD) (see shell.nix), so if the checkout path contains a space the emitted --suppressions=, --log-file=, and the final postgres invocation break (unterminated arguments / wrong binary path). Quote these when writing the wrapper, e.g. --suppressions="$PG_SOURCE_DIR/src/tools/valgrind.supp" and "$bindir/postgres" "\$@".


📄 shell.nix (L264-L268)

The print_proc macro dereferences $proc->waiting, but PGPROC (src/include/storage/proc.h) has no waiting member — that boolean field was removed; the current fields are lwWaiting and the ProcWaitStatus waitStatus enum. This macro will fail to evaluate in GDB. (Same broken-GDB-macro class as the confirmed print_mcxt/print_relcache findings.) High confidence.


📄 shell.nix (L233-L234)

The print_slot macro reads $slot->tts_empty and $slot->tts_shouldFree, but TupleTableSlot (src/include/executor/tuptable.h) has no such members. Those states live in the tts_flags bitmask (TTS_FLAG_EMPTY/TTS_FLAG_SHOULDFREE, exposed via TTS_EMPTY()/TTS_SHOULDFREE()), so this macro will fail to evaluate in GDB. (Same broken-GDB-macro class as the confirmed print_mcxt/print_relcache findings.) High confidence.


📄 shell.nix (L217-L218)

The print_tupdesc macro indexes $desc->attrs[$i], but TupleDescData (src/include/access/tupdesc.h) has no attrs member. The attribute array is no longer an embedded attrs field; Form_pg_attribute entries are stored after the struct and accessed via TupleDescAttr(desc, i) / TupleDescAttrAddress(), while the struct only exposes compact_attrs. This macro will fail to evaluate in GDB. (Same broken-GDB-macro class as the confirmed print_mcxt/print_relcache findings.) High confidence.


📄 src/test/regress/pg_regress.c (L1246-L1246)

Changing execl to execlp makes the regression driver resolve the shell via $PATH whenever shellprog lacks a slash. In practice SHELLPROG is always an absolute path ($(SHELL) in GNUmakefile/ecpg, hardcoded /bin/sh in meson.build), so this has no effect for the normal build and is a no-op there. It only matters if someone overrides SHELLPROG with a bare name, in which case it introduces a PATH-dependent shell lookup -- a footgun that makes test execution depend on the ambient environment and a potential hijack vector. This hunk is unrelated to any stated feature; for an upstream-bound patch it is unneeded churn and should be dropped unless the PR specifically justifies and documents the need (e.g. non-standard install prefixes). Confidence: high.

💡 Suggested change

Before:

		execlp(shellprog, shellprog, "-c", cmdline2, (char *) NULL);

After:

		execl(shellprog, shellprog, "-c", cmdline2, (char *) NULL);

📄 src/tools/pgindent/pgindent (L1-L1)

This makes pgindent the only Perl script in the entire tree using #!/usr/bin/env perl; every other script (~30+, e.g. genbki.pl, gen_node_support.pl, copyright.pl, mark_pgdllimport.pl, all generate-*.pl) uses #!/usr/bin/perl. For an upstream-destined PostgreSQL patch this is an inconsistent, unrelated hygiene change that violates minimal-diff discipline and breaks the project's established convention. If a PATH-based perl is needed for a local (e.g. Nix) environment, that belongs in the developer's environment, not in a committed shebang. Revert to keep it consistent with the rest of the tree. Confidence: high.

💡 Suggested change

Before:

#!/usr/bin/env perl

After:

#!/usr/bin/perl

📄 src/backend/storage/buffer/bufmgr.c (L4143-L4144)

Double-counting bug: a buffer demoted here always also passes the refcount==0 && COOL test immediately below, so SyncOneBuffer returns BUF_COOLED | BUF_REUSABLE (or BUF_COOLED | BUF_WRITTEN when dirty). In BgBufferSync the BUF_WRITTEN/BUF_REUSABLE increment and this standalone if (sync_state & BUF_COOLED) increment are not mutually exclusive, so the same physical buffer bumps reusable_buffers twice. That makes the LRU scan terminate roughly twice as early as upcoming_alloc_est intends, under-cleaning and under-cooling the pool exactly when pressure (cool_if_hot) is on. Count a cooled buffer only when it was not already counted as reusable/written.

💡 Suggested change

Before:

		if (sync_state & BUF_COOLED)
			reusable_buffers++;

After:

		else if (sync_state & BUF_COOLED)
			reusable_buffers++;

📄 src/backend/storage/buffer/freelist.c (L847-L848)

Trailing blank line added at EOF. git diff --check flags this as 'new blank line at end of file' - unrelated whitespace churn that will draw a review nit on -hackers. Remove it.


📄 src/backend/storage/buffer/bufmgr.c (L4241-L4243)

BUF_COOLED is set unconditionally after the lock is dropped and re-acquired. UnlockBufHdrExt() releases the header spinlock; between it and LockBufHdr() a concurrent PinBuffer() can pin and re-promote the buffer. The BUF_REUSABLE check below correctly re-reads the fresh state and will fail in that case, but BUF_COOLED is still returned and still counted toward reusable_buffers in BgBufferSync, overstating staged supply. Consider only reporting BUF_COOLED after confirming the re-locked state is still COOL/unpinned, or demote with a direct masked store without dropping the lock to avoid the race window entirely.


📄 src/backend/storage/buffer/freelist.c (L340-L341)

The numCoolClaims counter is incremented before the CAS that actually claims the buffer. If the CAS fails (a concurrent backend touched the buffer, which is exactly the HOT-pressure scenario where contention is likely), the counter was already bumped but no HOT buffer was claimed, and the loop retries, bumping it again. This inflates the metric that BgBufferSync() reads via StrategyCoolClaims() to decide cool_if_hot, so a burst of CAS retries can falsely signal eviction pressure. Increment only on CAS success, inside the if block (next to TrackNewBufferPin), mirroring how the function returns a buffer only after a successful CAS.

(moderate confidence)

💡 Suggested change

Before:

				pg_atomic_fetch_add_u64(&StrategyControl->numCoolClaims, 1);
				local_buf_state += BUF_REFCOUNT_ONE;

After:

				local_buf_state += BUF_REFCOUNT_ONE;

📄 src/include/storage/buf_internals.h (L191-L192)

This rationale is now inaccurate. The pin fast paths no longer reference BM_MAX_USAGE_COUNT: PinBuffer() (bufmgr.c) and PinLocalBuffer() (localbuf.c) both test BUF_STATE_GET_COOLSTATE(buf_state) < BUF_COOLSTATE_HOT directly. The actual remaining consumer of this macro is contrib/pg_buffercache (array sizing). Fix the comment to describe what truly keeps the macro alive, or the next reader will chase a pin fast path that doesn't use it.

(high confidence)

Comment thread .gdbinit
Comment on lines +45 to +46
# Create augmented tuple with embedded modified-column bitmap
break heap_hot_indexed_create_tuple

ghost Oct 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This entire file is developer-local scaffolding and should not be committed to a patch destined for pgsql-hackers/commitfest. It is unrelated to the HOT-indexed-updates feature itself and violates the minimal-diff discipline. Beyond that, it is broken as committed: the function-name breakpoints reference symbols that do not exist in this tree. heap_hot_indexed_create_tuple, heap_hot_indexed_serialize_bitmap, heap_xlog_indexed_update, etc. return no matches in the source, so GDB will reject these break commands as unresolved. Keep this in a local, un-tracked file (e.g. via .gitignore) instead.

Comment thread .gdbinit
Comment on lines +27 to +29
break heapam.c:4019
break heapam.c:4024
break heapam.c:4033

ghost Oct 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hardcoded absolute line-number breakpoints (heapam.c:4019, heapam.c:4024, heapam.c:4033, pruneheap.c:1802, indexam.c:299, execIndexing.c:370, etc.) are extremely fragile: they silently drift to the wrong statement the moment any of these files change, which is a footgun during review/debugging. If this file is kept at all, prefer function-name + offset breakpoints over raw line numbers so they survive edits.

Comment thread shell.nix
Comment on lines +2 to +5
pkgs,
pkgs-unstable,
system,
}: let

ghost Oct 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The system argument is destructured here but never referenced anywhere in the file (it only appears in comments/strings). Drop it from the function header to avoid an unused parameter, or use it where intended.

Comment thread shell.nix
if $context
printf "MemoryContext: %s\n", $context->name
printf " type: %s, parent: %p\n", $context->methods->name, $context->parent
printf " total: %zu, free: %zu\n", $context->mem_allocated, $context->freep - $context->freeptr

ghost Oct 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

print_mcxt dereferences $context->freep and $context->freeptr, but MemoryContextData (src/include/nodes/memnodes.h) has no such fields. The macro will fail to evaluate in GDB. The only size-related field available is mem_allocated; there is no free-pointer pair to subtract.

Comment thread shell.nix
define print_relcache
set $rel = (Relation)$arg0
if $rel
printf "Relation: %s.%s (OID: %u)\n", $rel->rd_rel->relnamespace, $rel->rd_rel->relname.data, $rel->rd_id

ghost Oct 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rd_rel->relnamespace is an OID (a 32-bit unsigned integer), not a string, so printing it with %s will dereference garbage. Use %u for the namespace OID.

Comment thread pg-aliases.sh
echo "Error: pgindent not found at $PG_SOURCE_DIR/src/tools/pgindent/pgindent"
else

modified_files=$(git diff --name-only "${since}" | grep -E "\.c$|\.h$")

ghost Oct 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

modified_files is assigned without local, so it leaks into the caller's interactive shell as a global. The sibling helpers pg-tidy/pg-spell correctly declare local files. Declare it local here too.

Comment thread shell.nix
Comment on lines +162 to +173
define print_list
set $list = (List*)$arg0
if $list
printf "List length: %d\n", $list->length
set $cell = $list->head
set $i = 0
while $cell && $i < $list->length
printf " [%d]: ", $i
print_node $cell->data.ptr_value
set $cell = $cell->next
set $i = $i + 1
end

ghost Oct 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The print_list GDB macro uses the pre-v13 cons-cell List layout: $list->head, $cell->next, and $cell->data.ptr_value. Since PostgreSQL 13 the List struct (src/include/nodes/pg_list.h) is array-based — it has elements/initial_elements and length/max_length, with no head field; ListCell is now a union with ptr_value directly (no .data sub-struct) and no next pointer. This macro will fail to evaluate against the tree it ships with. (Same broken-GDB-macro class as the confirmed print_mcxt/print_relcache findings.) High confidence.

Comment thread paper/Makefile Outdated
Comment on lines +1 to +3
# Requires: texlive-latex-recommended texlive-latex-extra texlive-publishers
# texlive-fonts-recommended texlive-fonts-extra cm-super
# texlive-bibtex-extra texlive-pictures latexmk

ghost Oct 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[high confidence] This entire paper/ directory (a LaTeX/texlive build for an ACM paper) is out of scope for a PostgreSQL patch. The change is destined for pgsql-hackers/commitfest, where the top rejection reason is a non-minimal diff that "does more than one thing." A paper build system has nothing to do with the clock-sweep buffer manager code change and should be removed from the patch (keep it in a separate repo/branch). Everything below is secondary to this.

Comment thread paper/Makefile Outdated
Comment on lines +9 to +10
clean:
latexmk -C

ghost Oct 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[high confidence] If this Makefile is kept at all, it does not follow PostgreSQL makefile conventions: it does not include $(top_builddir)/src/Makefile.global, uses no standard PG variables, and provides only clean (missing distclean and maintainer-clean). latexmk -C also removes the final PDF, so there is no separation between intermediate aux files and the committed/distributed output. It is additionally not wired into any parent Makefile or meson.build, making it orphaned scaffolding.

Comment thread paper/Makefile Outdated
Comment on lines +7 to +8
bounded-clock-sweep.pdf: bounded-clock-sweep.tex refs.bib $(wildcard *.dat)
latexmk -pdf -interaction=nonstopmode $<

ghost Oct 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[moderate confidence] The LaTeX build produces artifacts (bounded-clock-sweep.pdf, .aux, .log, .bbl, .blg, .fls, .fdb_latexmk, .out) that are not covered by the top-level .gitignore and there is no paper/.gitignore. These will show up as untracked files after a build and risk being accidentally committed.

Greg Burd added 4 commits October 7, 2026 13:26
Squashes the four CI commits this fork carried on top of upstream into
one, and cuts the carry from 3776 lines to ~1000.

Removed:

- .github/{README,QUICKSTART}.md and .github/docs/* (2273 lines). Beyond
  being bulk we rebase hourly, they had drifted into being wrong: they
  described a daily (not hourly) sync, Claude 3.5 Sonnet via the direct
  Anthropic API (we run Opus on Bedrock), a static IAM user (we use
  OIDC), and a Windows dependency-builder workflow that the same commit
  which documented it had already deleted. 16 referenced paths did not
  exist. An AI reviewer reading this repo ingests that as truth.

- sync-upstream-manual.yml (252 lines). sync-upstream.yml already has
  workflow_dispatch; the sole behavioral difference was a force_push
  toggle whose "false" setting cannot work after a rebase anyway.

- .github/.gitignore, which only ignored scripts/ai-review/, a directory
  that does not exist here.

Rewritten:

- sync-upstream.yml -> fork-sync-upstream.yml, 260 lines to 47. git
  rebase already handles the fast-forward, no-op and diverged cases that
  the ahead/behind arithmetic hand-rolled, so all that goes away, as do
  the issue open/comment/close machinery (a scheduled-run failure is
  already emailed to the one consumer, and the run log carries what the
  issue body would) and the "dev setup|dev v[0-9]" commit allowlist,
  which matched zero commits on master and had since it was written.
  Sync drops to every three hours: hourly meant 24 full-history clones
  of an 840MB repo per day, each force-push also re-triggering pg-ci.yml
  on master, to collect a handful of upstream commits.

Renamed:

- ocr-review.yml, ocr-model-check.yml -> fork-*.yml. Upstream owns
  .github/ too (it has touched pg-ci.yml three times this year), so
  fork-owned workflows now live under a name prefix upstream will not
  collide with, making the rebase structurally conflict-free and letting
  the sync guard match fork-owned paths exactly rather than allowing all
  of .github/.

Unchanged: the OCR review workflows' logic and .github/ocr/* (rule.json,
context.md, litellm.yaml, pg-history.py) - the part that carries value.
The OCR review workflows and their config were the last bulk this fork
carried on top of upstream: ~970 lines rebased onto postgres/postgres
every few hours, editable only by amending a commit that gets force-
pushed. They are now reusable (workflow_call) workflows in the sidecar
repo gburd/ci-workflows, and master carries two caller stubs instead.

Carry after this: 3 files, ~110 lines, one commit.

The sidecar is also where the value is: ocr/rule.json and ocr/context.md
(the PostgreSQL committer-grade review corpus) become normal reviewable
files with normal PRs and history, rather than a blob inside a commit
that is rewritten on every sync.

Mechanics worth knowing:

- The review jobs check ci-workflows out at github.job_workflow_sha, so
  the rules always come from the same commit as the logic using them, and
  pinning @v1 (or a sha) in the stub pins both. This replaces the old
  "git show origin/<default_branch>:.github/ocr/..." dance, which existed
  only because the config had to be read off a branch other than the PR's.

- AWS settings are passed as explicit inputs rather than read from vars
  inside the reusable workflow. vars does resolve across the boundary,
  but the contract stays visible at the call site.

- A reusable workflow does not change the OIDC subject: it stays
  repo:gburd/postgres:*, so the ocr-bedrock-ci trust policy is unchanged.

- ci-workflows must stay public; callers read it with GITHUB_TOKEN.

The sync guard narrows to .github/workflows/fork-* accordingly, and
sync-upstream keeps needing SYNC_TOKEN: the stubs are still workflow
files, which GITHUB_TOKEN may not push.
This isn't a commit that will be submitting for review, it is purely for
local developer tooling while developing this patch set.  Ignore it.
A buffer becomes a candidate for eviction only once its usage_count has been
decremented to zero, and PinBuffer() saturates the count on every access.  With
a pass time of T, a buffer must therefore go roughly BM_MAX_USAGE_COUNT * T
without being touched before the sweep can take it.  T grows with NBuffers, so
on a large pool a workload that touches much of the pool more often than that
re-promotes buffers faster than the hand decrements them.  The population of
candidates collapses, and because a decrement counts as progress and resets
trycounter, StrategyGetBuffer() has no bound on the work it will perform: a
single buffer allocation can sweep the entire pool.

The header has documented the possibility for years -- "it can take as many as
BM_MAX_USAGE_COUNT+1 complete cycles of the clock-sweep hand to find a free
buffer" -- but it is reached in practice, not only in theory, and the cost
scales linearly with shared_buffers.

Bound it.  A sweep that has decremented BUF_DECREMENT_CLAIM_THRESHOLD buffers
without finding a single one at zero has established, by observation rather
than by guesswork, that the pool is hotter than the hand can grind down.  From
that point in the same call it claims the next unpinned buffer directly instead
of decrementing it.  usage_count is cleared when a victim is reused
(InvalidateVictimBuffer), so the claim needs no separate step to drive the
count to zero.

The cost is evicting a buffer that has not been passed over the full
BM_MAX_USAGE_COUNT times.  Both sides of that trade are measured: the
worst-case advances an allocation performs is the threshold plus about two, and
on a workload with a hot set re-read by most queries a threshold of 4 evicts
thousands of hot pages for 1.8 points of hot-set hit ratio, 32 evicts about ten
for 0.03 points, and 128 evicts none for no measurable cost.  The default of
128 therefore bounds the tail four orders of magnitude below an unbounded scan
without measurably degrading replacement quality.  A control loop that adapted
the threshold was tried and proved worse than a fixed value on both axes.  The counter is call-local, so the common path takes no additional
atomic.  Under a workload the hand can keep up with, the threshold is never
reached and behaviour is exactly as before.

StrategyControl gains numForcedClaims, counting allocations that had to resort
to this.  A growing value is the signal that the pool is undersized for its
access rate.

This changes only the termination rule of the sweep.  It does not alter the
0..5 usage_count, the meaning of any buffer state bit, the background writer,
or the BufferAccessStrategy rings, and it is independent of any change to the
replacement state encoding.

ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 OCR found 16 issue(s).

  • 14 inline, 2 in summary

📄 .clangd

.clangd is personal editor/LSP configuration and should not be committed to a PostgreSQL patch. Additionally, the hardcoded relative include -I../../../../src/include assumes a specific nested working directory and will not resolve from the repo root or for most translation units, defeating the config's purpose. Recommend removing this file from the patch (keep it git-ignored locally).


📄 src/include/storage/buf_internals.h

The claim "The worst-case advances an allocation performs is almost exactly this threshold plus two" is only true under the targeted hot-pool workload. The true upper bound on advances per allocation remains NBuffers: after the threshold is reached, the sweep still iterates over pinned buffers (refcount != 0 path decrements trycounter without claiming) and over usage_count==0 buffers whose CAS loses to a concurrent pinner, until trycounter hits zero. Recommend qualifying this as an observed/typical worst case rather than an absolute one, to avoid misleading a future reader into assuming a hard threshold+2 bound.

@@ -1,4 +1,4 @@
#!/usr/bin/perl
#!/usr/bin/env perl

ghost Oct 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shebang change is an unrelated, non-minimal diff that deviates from the project-wide convention. Every other Perl script in the tree (e.g. src/tools/copyright.pl, mark_pgdllimport.pl, src/backend/catalog/genbki.pl, and ~40 others) uses #!/usr/bin/perl; this makes pgindent the sole outlier. PostgreSQL's build system locates the Perl interpreter explicitly (via $(PERL) / perl.path), so /usr/bin/env perl provides no benefit here and only introduces inconsistency. Unless the surrounding change actually requires a different interpreter resolution, revert this to keep the diff minimal and consistent with the rest of the tree.

Suggested change
#!/usr/bin/env perl
#!/usr/bin/perl

Comment thread .gdbinit
@@ -0,0 +1,156 @@
# HOT Indexed Updates — GDB breakpoints for code review

ghost Oct 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This entire file is developer-local debugging state that does not belong in a patch destined for pgsql-hackers. Beyond the minimal-diff objection, it is actively broken: the breakpoints hardcode absolute source line numbers (heapam.c:4019, pruneheap.c:1802, pruneheap.c:2936, etc.) and symbol names (heap_hot_indexed_create_tuple, heap_hot_search_buffer, ...) that do not exist in this tree. A codebase search for heap_hot_indexed_ returns zero matches, and the referenced heap/pruneheap changes are not part of this change set. These breakpoints point at nothing and will rot silently the moment the source shifts by a line. Recommend removing this file from the patch and keeping it in a personal, git-ignored location.

Comment thread .gdbinit
@@ -0,0 +1,156 @@
# HOT Indexed Updates — GDB breakpoints for code review

ghost Oct 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-ASCII characters (em-dash —) appear in this file. PostgreSQL requires ASCII-only in source and committed files.

Comment thread pg-aliases.sh
alias pg-full-clean='trash "$PG_BUILD_DIR" "$PG_INSTALL_DIR" 2>/dev/null || rm -rf "$PG_BUILD_DIR" "$PG_INSTALL_DIR"; echo "Build and install directories cleaned"'

# Database management
alias pg-init='trash "$PG_DATA_DIR" 2>/dev/null || rm -rf "$PG_DATA_DIR"; "$PG_INSTALL_DIR/bin/initdb" --debug --no-clean "$PG_DATA_DIR"'

ghost Oct 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rm -rf "$PG_DATA_DIR" (and pg-full-clean's rm -rf "$PG_BUILD_DIR" "$PG_INSTALL_DIR") are a data-loss footgun when the variable is unset/empty. If this file is sourced outside the Nix shellHook that exports these vars, rm -rf "" degrades to deleting relative to the current directory. Guard each destructive deletion with a non-empty check (e.g. [ -n "$PG_DATA_DIR" ] && ...). More fundamentally, this personal alias library does not belong in a pgsql-hackers patch.

Comment thread pg-aliases.sh
if [ "$last_compiler" != "$current_compiler" ] && [ "$last_compiler" != "unknown" ]; then
echo "Detected compiler change from $last_compiler to $current_compiler"
echo "Cleaning build directory..."
trash "$build_dir" 2>/dev/null || rm -rf "$build_dir"

ghost Oct 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

trash is not a POSIX/coreutils tool and is not provided by getPostgreSQLDeps in shell.nix, so on a clean dev shell the || rm -rf fallback is always taken. Either add trash/trash-cli to the Nix buildInputs or drop the trash indirection to keep the documented workflow reproducible.

* StrategyGetBuffer). A growing value means the pool is hotter than the
* clock hand can keep up with.
*/
pg_atomic_uint64 numForcedClaims;

ghost Oct 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

numForcedClaims is write-only dead instrumentation. Unlike its sibling numBufferAllocs, which is consumed by StrategySyncStart() (feeding the bgwriter) and reset there, numForcedClaims is only initialized (StrategyCtlShmemInit) and incremented (StrategyGetBuffer) but has no reader anywhere in the tree. It is therefore not observable, contradicting the comment's premise ("A growing value means the pool is hotter than the clock hand can keep up with") -- nothing can see that value grow. Either wire it up to a consumer (e.g. pg_stat_bgwriter / a pg_buffercache summary / StrategySyncStart out-param) with tests, or drop the field and its atomic add from the hot path. As-is it bloats the shared BufferStrategyControl struct and adds a cross-core atomic on the pressure path for no benefit (YAGNI).

* when the victim is reused (see InvalidateVictimBuffer), so
* there is no need to drive it to zero first.
*/
pg_atomic_fetch_add_u64(&StrategyControl->numForcedClaims, 1);

ghost Oct 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This atomic add sits on the eviction pressure path and feeds a counter that is never read (see numForcedClaims field). It is pure overhead: a 64-bit RMW on the StrategyControl cache line (shared with numBufferAllocs/completePasses) under exactly the contended, buffer-starved workload this feature targets. If the counter is kept, it should at least be justified by an observable consumer; otherwise remove this line along with the field.

Comment on lines +170 to +171
* A control loop that adapted this value was tried and was worse than a fixed
* one on both axes; see the commit message for that experiment.

ghost Oct 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"see the commit message for that experiment" is a fragile cross-reference: once committed/squashed/rebased the commit message is not accessible from the source tree, and readers of the code can't follow it. PostgreSQL comments should be self-contained. Either state the conclusion inline (adaptive control was tried and performed worse on both axes, so a fixed threshold is used) and drop the pointer, or reference the pgsql-hackers thread Message-Id. Also, this block reads as a benchmark narrative; it explains WHY well but is unusually long for buf_internals.h -- consider trimming the per-threshold measurement figures (4/32/128) to the essential conclusion.

Comment on lines +325 to +327
if (decremented < BUF_DECREMENT_CLAIM_THRESHOLD)
{
local_buf_state -= BUF_USAGECOUNT_ONE;

ghost Oct 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This changes the buffer-eviction policy under pressure (claiming a buffer with non-zero usage_count after BUF_DECREMENT_CLAIM_THRESHOLD fruitless decrements), yet the patch adds no regression/TAP coverage for the new path. A user-visible behavior change in the clock-sweep victim selection needs a test that exercises the forced-claim branch (e.g. a small NBuffers pool kept hot enough that the threshold is reached), otherwise this is WIP, not commit-ready. Confidence: moderate.

Comment on lines +158 to +168
* The value trades the tail against hit ratio, and both sides have been
* measured. The worst-case advances an allocation performs is almost exactly
* this threshold plus two, so the tail is a direct and predictable function of
* it. The cost is that a claimed buffer is evicted without having been passed
* over the full number of times: on a workload with a hot set re-read by most
* queries, a threshold of 4 evicts thousands of genuinely hot pages and costs
* 1.8 percentage points of hot-set hit ratio, a threshold of 32 evicts about
* ten and costs 0.03 points, and at 128 nothing hot is evicted and the cost is
* not measurable. 128 therefore buys a bound four orders of magnitude below an
* unbounded scan for no measurable loss of replacement quality, which is the
* right place to sit.

ghost Oct 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This header comment embeds very specific, unverifiable benchmark numbers ("1.8 percentage points", "0.03 points", "four orders of magnitude") with no reference to a reproducible benchmark or pgsql-hackers thread/Message-Id. PostgreSQL comments should explain the invariant/why, not carry a results table that will silently rot as the code and workloads change. Trim this to the design rationale (bounded work vs. replacement quality) and move the measurements to the commit message / -hackers thread. Confidence: moderate.

Confirms the clock-sweep tail across three pool sizes on storage fast enough to
expose it: Fedora 44, XFS on RAID-0 instance-local NVMe at 429k random read
IOPS, no network-attached storage in the path.

The stock worst case is exactly one full pass of the buffer pool at every size
measured (N+5, N+5, N+38; tail/N = 1.0000 to five figures), so it scales
directly with shared_buffers.  The bound holds it flat at 130-134 advances
across an eightfold pool range, a 3,971x / 7,825x / 32,017x reduction, with the
factor growing precisely because the bounded quantity is the one that scales.

Cost is nil on the other two axes: throughput within 0.4%, amplification within
0.019 advances per victim, hit ratio within 0.0014 points, all inside
run-to-run variation.

Also records why an earlier attempt on a 6,000 IOPS volume measured nothing:
when every miss waits on storage the allocation rate is throttled and the clock
hand keeps up, so fast local storage is a precondition for the regime
regardless of skew or working-set size.  And that several hundred thousand
buffers at usage_count zero -- true in every arm, including stock -- is not
evidence the tail is absent.

Data in .agent/nvme-2026-10-07/ (not committed).

ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 OCR found 26 issue(s).

  • 25 inline, 1 in summary (inline capped at 25)

📄 pg-aliases.sh (L508-L508)

modified_files is assigned without local, so it leaks into the caller's shell environment. Since this file is sourced into interactive shells (see shell.nix source ./pg-aliases.sh), the function clobbers any pre-existing modified_files variable the user may have and leaves it set afterwards. Every other helper here correctly declares its working variables local. Declare it local (ideally at the top of the function alongside since).

💡 Suggested change

Before:

		modified_files=$(git diff --name-only "${since}" | grep -E "\.c$|\.h$")

After:

		local modified_files
		modified_files=$(git diff --name-only "${since}" | grep -E "\.c$|\.h$")

Comment thread .gdbinit
@@ -0,0 +1,156 @@
# HOT Indexed Updates — GDB breakpoints for code review

ghost Oct 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This developer-only .gdbinit does not belong in a PostgreSQL patch and would be rejected on pgsql-hackers as unrelated noise. Worse, it is scaffolding for an unmerged feature: the breakpoints reference heap_hot_indexed_create_tuple, heap_hot_indexed_serialize_bitmap, heap_xlog_indexed_update, XLOG_HEAP2_INDEXED_UPDATE, and indexed_attrs (indexam.c) — none of which exist anywhere in this tree (verified via code search, zero matches). A source .gdbinit against today's binary will fail to set these breakpoints. Keep this in a developer's local, git-ignored environment, not in the source tree. [high confidence]

Comment thread .gdbinit
Comment on lines +19 to +20
# Main entry: heap_update
break heapam.c:3210

ghost Oct 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Breakpoints pinned to absolute source line numbers are brittle and have already drifted. heapam.c:3210 is the closing brace of heap_delete — heap_update actually begins at line 3269. The accompanying prose ("Line 4019: pure HOT", "Line 4024: HOT indexed path", etc.) will desync from the file on any edit, making this file actively misleading. Function-name breakpoints (as used elsewhere in this file) are the only maintainable form; the numeric ones should be dropped. [high confidence]

Comment thread .clangd
Comment on lines +7 to +8
CompileFlags:
CompilationDatabase: build/ # Search build/ directory for compile_commands.json

ghost Oct 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a per-developer IDE environment file (CompilationDatabase: build/, personal flags like -DLOCAL, -O2, -std=c11, and hard-coded relative includes -I. / -I../../../../src/include). Committing it imposes one contributor's local layout on everyone and is unrelated to any functional change in this series — it would be flagged as non-minimal diff on pgsql-hackers. Such IDE/build artifacts belong in a personal, git-ignored config, not the source tree. [high confidence]

Comment on lines +8 to +10
name: OCR AI Review

on:

ghost Oct 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This job declares no permissions block, so GITHUB_TOKEN defaults to the repo's broad scope. The sibling workflows pg-ci.yml and fork-sync-upstream.yml both pin permissions: contents: read; this one does not. That is especially risky here because the issue_comment trigger (lines 14-15) lets anyone who can comment on a PR invoke a review run. Add an explicit least-privilege permissions block at the top level (or on the job) so the caller only grants what the reusable workflow actually needs.

Suggested change
name: OCR AI Review
on:
name: OCR AI Review
permissions:
contents: read
pull-requests: write
on:


on:
pull_request:
# Note: no draft filter — drafts are reviewed too.

ghost Oct 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ASCII-only is required in source. This comment uses an em-dash (U+2014). Replace it with an ASCII hyphen/dash.

Suggested change
# Note: no draft filter — drafts are reviewed too.
# Note: no draft filter - drafts are reviewed too.

Comment thread pg-aliases.sh

alias pg-start='ulimit -c unlimited && "$PG_INSTALL_DIR/bin/postgres" -D "$PG_DATA_DIR" -k "$PG_DATA_DIR"'

alias pg-stop='pkill -f "postgres.*-D.*$PG_DATA_DIR" || true'

ghost Oct 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pg-init and pg-full-clean fall back to rm -rf "$PG_DATA_DIR" / rm -rf "$PG_BUILD_DIR" "$PG_INSTALL_DIR". These are safe only because shell.nix always exports these vars; if this script is sourced standalone (its only guard is the warning in shell.nix, not the script itself) the vars are empty and these expand to rm -rf "" (harmless) but pg-stop/pg-status build the pkill -f "postgres.*-D.*$PG_DATA_DIR" pattern which, with an empty var, becomes postgres.*-D.* and kills every postgres on the host. Add a non-empty guard on PG_DATA_DIR before the destructive/pkill operations so the script fails safe outside the nix shell.

Comment thread pg-aliases.sh
--dlpath="$bdir/src/test/regress" \
--outputdir="$outdir" \
--temp-instance="$outdir/tmp" \
--port=40099 \

ghost Oct 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hardcoded --port=40099 means two concurrent pg-test-one invocations (or anything already bound to 40099) collide and fail non-obviously. Consider deriving the port from the PID/PWD or letting pg_regress pick a free port, to avoid surprising failures on a shared dev box or parallel runs.

@@ -1,4 +1,4 @@
#!/usr/bin/perl
#!/usr/bin/env perl

ghost Oct 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This changes pgindent's shebang to #!/usr/bin/env perl, making it the only Perl script in the entire tree that doesn't use #!/usr/bin/perl. All other ~35 Perl scripts/tools (genbki.pl, gen_node_support.pl, copyright.pl, mark_pgdllimport.pl, etc.) use the fixed #!/usr/bin/perl path. This one-line deviation is unrelated to any functional change in this PR, breaks tree-wide consistency, and will be rejected on the minimal-diff / consistency grounds the project enforces. If a #!/usr/bin/env convention were desired it would need to be a separate, tree-wide discussion on -hackers, not a single-file change. Revert to keep the diff minimal and consistent. (high confidence)

Suggested change
#!/usr/bin/env perl
#!/usr/bin/perl

* StrategyGetBuffer). A growing value means the pool is hotter than the
* clock hand can keep up with.
*/
pg_atomic_uint64 numForcedClaims;

ghost Oct 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

numForcedClaims is write-only dead state. It is incremented here and initialized in StrategyCtlShmemInit, but no code anywhere in the tree ever reads or exports it (only the .tex paper references the name), and unlike its sibling numBufferAllocs it is never reset or returned by StrategySyncStart. It therefore just enlarges the shared BufferStrategyControl struct and adds an atomic RMW on the hot allocation path for no observable benefit. Either wire it into an observable path (e.g. a pg_stat_bgwriter-style counter with matching reset/read handling) or drop it. Minimal patches should not leave unused scaffolding. [high confidence]

Comment on lines +344 to +345
pg_atomic_fetch_add_u64(&StrategyControl->numForcedClaims, 1);
local_buf_state += BUF_REFCOUNT_ONE;

ghost Oct 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The counter is incremented before the CAS below, so it counts claim attempts, not successful forced claims. When the CAS at the following line fails (concurrent modification of buf->state -- precisely the contended/hot-pool scenario this path is designed for), the inner for (;;) re-reads old_buf_state and re-enters this branch, incrementing numForcedClaims again without any buffer having been claimed. Move the increment to inside the successful-CAS block so it tracks actual forced claims. [medium confidence]

Suggested change
pg_atomic_fetch_add_u64(&StrategyControl->numForcedClaims, 1);
local_buf_state += BUF_REFCOUNT_ONE;
local_buf_state += BUF_REFCOUNT_ONE;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant