Thread

Re: eliminate xl_heap_visible to reduce WAL (and eventually set VM on-access)

Kirill Reshke <reshkekirill@gmail.com> — 2025-12-18T15:45:58Z

On Thu, 18 Dec 2025 at 20:18, Melanie Plageman
<melanieplageman@gmail.com> wrote:
> > Also, after the whole set is committed, we should then never
> > experience discrepancy between  PD_ALL_VISIBLE and VM bits? Because
> > they will be set in a single WAL record. The only cases when heap and
> > VM disagrees on all-visibility then are corruption,
> > pg_visibilitymap_truncate and old data (data before v19+ upgrade?)
> > If my understanding is correct, should we add document this?
>
> Even on current master, I don't see a scenario other than VM
> corruption or truncation where PD_ALL_VISIBLE can be set but not the
> VM (or vice versa). The only way would be if you error out after
> setting PD_ALL_VISIBLE before setting the VM. Setting PD_ALL_VISIBLE
> is not in a critical section in lazy_scan_prune(), so it won't panic
> and dump shared memory, so the buffer with PD_ALL_VISIBLE set may
> later get written out. But the only obvious way I see to error out of
> MarkBufferDirty() is if the buffer is not valid -- which would have
> kept us from doing previous operations on the buffer, I would think.
>

Well... I may be missing something, but on current HEAD,
XLOG_HEAP2_PRUNE_VACUUM_SCAN and XLOG_HEAP2_VISIBLE are two different
record, XLOG_HEAP2_PRUNE_VACUUM_SCAN being always emitted first. So,
WAL writer may end up kill-9-ed just after
XLOG_HEAP2_PRUNE_VACUUM_SCAN makes it to the disk, and
XLOG_HEAP2_VISIBLE never. Crash recovery then, and we have
discrepancy. This does not happen with a single WAL record.
Another simple reproducer here: standby streaming, receiving
XLOG_HEAP2_PRUNE_VACUUM_SCAN from primary, Then network becomes bad,
and we never get XLOG_HEAP2_VISIBLE from primary. Then we promoted by
the admin. And again, VM bit vs PD_ALL_VISIBLE discrepancy. Am I
missing something?


-- 
Best regards,
Kirill Reshke