Re: Tepid: selective index updates for heap relations
Greg Burd <greg@burd.me>
From: "Greg Burd" <greg@burd.me>
To: pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2026-07-06T13:16:22Z
Lists: pgsql-hackers
Attachments
- v56-0001-Add-tests-to-cover-a-variety-of-heap-HOT-update-.patch (text/x-patch) patch v56-0001
- v56-0002-Identify-modified-indexed-attributes-in-the-exec.patch (text/x-patch) patch v56-0002
- v56-0003-Add-the-HOT-indexed-on-disk-format-inline-attr-b.patch (text/x-patch) patch v56-0003
- v56-0004-Add-HOT-indexed-updates-selective-index-maintena.patch (text/x-patch) patch v56-0004
- v56-0005-Collapse-dead-HOT-indexed-chains-to-xid-free-stu.patch (text/x-patch) patch v56-0005
- v56-0006-Teach-amcheck-to-recognize-HOT-indexed-chains-an.patch (text/x-patch) patch v56-0006
- v56-0007-Add-HOT-indexed-statistics-and-the-comprehensive.patch (text/x-patch) patch v56-0007
- v56-0008-Gate-HOT-indexed-updates-on-the-logical-replicat.patch (text/x-patch) patch v56-0008
- v56-0009-DO-NOT-MERGE-Add-a-HOT-SIU-benchmark-harness.patch (text/x-patch) patch v56-0009
Hello, I've rebased Tepid (e994f956e48) and done a bit of minor cleanup. No new features, no design changes. Same nine commits, same structure. Two commits are unchanged (the HOT-behavior test commit and the benchmark harness). * pg_surgery now understands SIU stubs. heap_force_freeze/heap_force_kill skip xid-free HOT-indexed stubs instead of erroring or corrupting them, with a regression test that builds a real stub. (This was the one substantive gap the macOS reviewer flagged against v48.) * Prune/vacuum correctness — the most important fixes: - Restored upstream's hard elog(ERROR, "dead heap-only tuple … not linked to from any HOT chain") guard for the classic-HOT case. v48 silently reclaimed such a tuple, which would have masked real heap corruption; SIU's legitimate stub handling is now cleanly separated from that error path. - Fixed a primary/standby divergence bug: the lazy_vacuum_heap_page WAL record must always log a hardcoded false cleanup-lock flag (that call site never re-points redirects/stubs), and v48 was passing the runtime-variable value. - Added bounds checking (Assert(bmnatts <= relnatts) plus a defensive clamp) before the stub-bitmap union/subset operations, closing a stack-buffer-overflow risk from a corrupt or unbounded stub natts. * Executor / index-scan: - Fixed an out-of-bounds read in ExecCompareSlotAttrs (missing continue after the system-attribute branch). - RelationGetIndexedAttrs now parses raw catalog text directly rather than going through RelationGetIndexExpressions/Predicate, which could const-fold away Var references and under-report indexed attributes. - Removed dead code (an unused idx_attrs fetch in simple_heap_update, an empty branch in index_delete_check_htid) and corrected an inaccurate "must mirror heap_multi_insert" comment. * amcheck now reports corruption on out-of-range or self-referential stub forward links, matching the existing redirect-link check. * Statistics: the n_hot_indexed counter and the matched/skipped pgstat counters now exclude stubs and fire only after the partial-index predicate check (v48 over-counted). * Logical replication: rebased over upstream's new conflict_log_destination feature (a subscription bit-flag collision resolved, 0x00080000->0x00100000), and the subscription field/option renamed hotindexedmode→hotindexedonapply for clarity, with a matching psql \dRs+ display fix (shows the word, not the raw char code). * Tests: hardened the count-sensitive tables against an autovacuum lock-steal race (autovacuum_enabled = false), and replaced a relpages-based assertion (only updated by VACUUM/ANALYZE, so trivially true) with a real pg_relation_size check. * Plus routine cleanups — removed a few redundant/unused includes, fixed typos. best. -greg