Horton

Re: eliminate xl_heap_visible to reduce WAL (and eventually set VM on-access)

Melanie Plageman <melanieplageman@gmail.com>

View thread
From: Melanie Plageman <melanieplageman@gmail.com>
To: Kirill Reshke <reshkekirill@gmail.com>
Cc: Xuneng Zhou <xunengzhou@gmail.com>, Andres Freund <andres@anarazel.de>, Robert Haas <robertmhaas@gmail.com>, Andrey Borodin <x4mmm@yandex-team.ru>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Heikki Linnakangas <hlinnaka@iki.fi>, Chao Li <li.evan.chao@gmail.com>
Date: 2025-12-22T18:20:59Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Remove table_scan_analyze_next_tuple unneeded parameter OldestXmin

  2. Simplify visibility check in heap_page_would_be_all_visible()

  3. Eliminate use of cached VM value in lazy_scan_prune()

  4. Combine visibilitymap_set() cases in lazy_scan_prune()

  5. Fix const qualification in prune_freeze_setup()

  6. Simplify vacuum visibility assertion

  7. Split heap_page_prune_and_freeze() into helpers

  8. Assert that cutoffs are provided if freezing will be attempted

  9. Split PruneFreezeParams initializers to one field per line

  10. Refactor heap_page_prune_and_freeze() parameters into a struct

  11. Make heap_page_is_all_visible independent of LVRelState

  12. Inline TransactionIdFollows/Precedes[OrEquals]()

  13. Add helper for freeze determination to heap_page_prune_and_freeze

  14. Bump XLOG_PAGE_MAGIC after xl_heap_prune change

  15. Correct prune WAL record opcode name in comment

  16. Add error codes when vacuum discovers VM corruption

  17. Remove unused xl_heap_prune member, reason

  18. Remove unneeded VM pin from VM replay

  19. Add assert and log message to visibilitymap_set

  20. Add error codes to some corruption log messages

Attachments

On Sat, Dec 20, 2025 at 7:32 AM Kirill Reshke <reshkekirill@gmail.com> wrote:
>
> Hi! I checked v29-0009, about HeapTupleSatisfiesVacuumHorizon. Origins
> of this code track down to fdf9e21196a6 which was committed as part of
> [0], at which point
> there was no HeapTupleSatisfiesVacuumHorizon function. I guess this is
> the reason this optimization was not performed earlier.

Thanks for taking a look into this!

> I also think this patch is correct, because we do similar things for
> HEAPTUPLE_DEAD & HEAPTUPLE_RECENTLY_DEAD, and
> HeapTupleSatisfiesVacuumHorizon is just a proxy to
> HeapTupleSatisfiesVacuumHorizon with only difference in DEAD VS
> RECENTLY_DEAD handling.
>
> Similar change could be done at heapam_scan_analyze_next_tuple
>
> ...
> case HEAPTUPLE_DEAD:
> case HEAPTUPLE_RECENTLY_DEAD:
> /* Count dead and recently-dead rows */
> *deadrows += 1;
> break;

In v30 sent here [1], I did end up making this change in 0010. I just
realized that I should have also changed
table_scan_analyze_next_tuple() and removed the call to
GetOldestRemovableTransactionId(). I've done that in attached v31.

I'm not sure we should change the table AM API (by removing
OldestXmin), though. I looked for table AMs implementing
scan_analyze_next_tuple() to see if they use OldestXmin. I found two:
OrioleDB [2] and Citus columnar [3], which both implement
scan_analyze_next_tuple() and neither of them use OldestXmin. I
couldn't easily find other table AMs implementing
scan_analyze_next_tuple(). I don't have a strong sense of whether or
not I should make this change. Changing it is churn to a public API
and doesn't specifically enable us to do something.

I could also just leave it unused by heapam's implementation. I
haven't checked what, if any, other table AMs callbacks have
parameters completely unused by their heap implementation.

So, I'm on the fence about whether or not to make the change at all,
and, if I do, whether or not to change the table AM callback. That is
done in v31, though, so we can discuss.

- Melanie

[1] https://www.postgresql.org/message-id/CAAKRu_ZCjHoRPfQ8AbMrFY8TOMCPAvZ0_m9SX7yg0edfTk45-g%40mail.gmail.com
[2] https://github.com/orioledb/orioledb/blob/acff65984d106dabf708a179e2c6694297e08c02/src/tableam/handler.c#L978C68-L978C78
[3] https://github.com/citusdata/citus/blob/ee3812d267db3ab007efb6f5f432c82c1f448695/src/backend/columnar/columnar_tableam.c#L1418