Re: Confine vacuum skip logic to lazy_scan_skip

Thomas Munro <thomas.munro@gmail.com>

From: Thomas Munro <thomas.munro@gmail.com>

To: Noah Misch <noah@leadboat.com>

Cc: Melanie Plageman <melanieplageman@gmail.com>, vignesh C <vignesh21@gmail.com>, Andres Freund <andres@anarazel.de>, Pg Hackers <pgsql-hackers@postgresql.org>, Heikki Linnakangas <hlinnaka@iki.fi>, Nazir Bilal Yavuz <byavuz81@gmail.com>, Robert Haas <robertmhaas@gmail.com>, "Andrey M. Borodin" <x4mmm@yandex-team.ru>

Date: 2024-07-24T05:40:12Z

Lists: pgsql-hackers

Attachments

v12-0001-Use-streaming-I-O-in-VACUUM-first-pass.patch (text/x-patch) patch v12-0001
v12-0002-Use-streaming-I-O-in-VACUUM-second-pass.patch (text/x-patch) patch v12-0002

On Tue, Jul 16, 2024 at 1:52 PM Noah Misch <noah@leadboat.com> wrote:
> On Mon, Jul 15, 2024 at 03:26:32PM +1200, Thomas Munro wrote:
> That's reasonable.  radixtree already forbids mutations concurrent with
> iteration, so there's no new concurrency hazard.  One alternative is
> per_buffer_data big enough for MaxOffsetNumber, but that might thrash caches
> measurably.  That patch is good to go apart from these trivialities:

Thanks!  I have pushed that patch, without those changes you didn't like.

Here's are Melanie's patches again.  They work, and the WAL flush
frequency problem is mostly gone since we increased the BAS_VACUUM
default ring size (commit 98f320eb), but I'm still looking into how
this read-ahead and the write-behind generated by vacuum (using
patches not yet posted) should interact with each other and the ring
system, and bouncing ideas around about that with my colleagues.  More
on that soon, hopefully.  I suspect that there won't be changes to
these patches as a result, but I still want to hold off for a bit.

Commits

Fix explicit valgrind interaction in read_stream.c.
- 57dca6faa9bd 17.5 landed
- 2a8a00674e97 18.0 landed
Reduce scope of heap vacuum per_buffer_data
- c623e8593ec4 18.0 landed
Use streaming read I/O in VACUUM's third phase
- c3e775e608f2 18.0 landed
Use streaming read I/O in VACUUM's first phase
- 9256822608f3 18.0 landed
Convert heap_vac_scan_next_block() boolean parameters to flags
- 32acad7d1d0a 18.0 landed
Refactor tidstore.c iterator buffering.
- f6bef362cac8 18.0 landed
Increase default vacuum_buffer_usage_limit to 2MB.
- 98f320eb2ef0 17.0 landed
Remove unneeded vacuum_delay_point from heap_vac_scan_get_next_block
- 3d8652cd3284 17.0 landed
Confine vacuum skip logic to lazy_scan_skip()
- 4e76f984a773 17.0 landed
Set all_visible_according_to_vm correctly with DISABLE_PAGE_SKIPPING
- 407cb6c6589f 16.3 landed
- 674e49c73c1c 17.0 landed
Tighten up VACUUM's approach to setting VM bits.
- 980ae173108e 16.0 cited