Horton

Re: Confine vacuum skip logic to lazy_scan_skip

Melanie Plageman <melanieplageman@gmail.com>

View thread View on postgr.es
From: Melanie Plageman <melanieplageman@gmail.com>
To: vignesh C <vignesh21@gmail.com>
Cc: Andres Freund <andres@anarazel.de>, Pg Hackers <pgsql-hackers@postgresql.org>, Heikki Linnakangas <hlinnaka@iki.fi>, Thomas Munro <thomas.munro@gmail.com>, Nazir Bilal Yavuz <byavuz81@gmail.com>, Robert Haas <robertmhaas@gmail.com>, "Andrey M. Borodin" <x4mmm@yandex-team.ru>
Date: 2024-03-10T16:31:12Z
Lists: pgsql-hackers 
On Wed, Mar 6, 2024 at 6:47 PM Melanie Plageman
<melanieplageman@gmail.com> wrote:
>
> Performance results:
>
> The TL;DR of my performance results is that streaming read vacuum is
> faster. However there is an issue with the interaction of the streaming
> read code and the vacuum buffer access strategy which must be addressed.

I have investigated the interaction between
maintenance_io_concurrency, streaming reads, and the vacuum buffer
access strategy (BAS_VACUUM).

The streaming read API limits max_pinned_buffers to a pinned buffer
multiplier (currently 4) * maintenance_io_concurrency buffers with the
goal of constructing reads of at least MAX_BUFFERS_PER_TRANSFER size.

Since the BAS_VACUUM ring buffer is size 256 kB or 32 buffers with
default block size, that means that for a fully uncached vacuum in
which all blocks must be vacuumed and will be dirtied, you'd have to
set maintenance_io_concurrency at 8 or lower to see the same number of
reuses (and shared buffer consumption) as master.

Given that we allow users to specify BUFFER_USAGE_LIMIT to vacuum, it
seems like we should force max_pinned_buffers to a value that
guarantees the expected shared buffer usage by vacuum. But that means
that maintenance_io_concurrency does not have a predictable impact on
streaming read vacuum.

What is the right thing to do here?

At the least, the default size of the BAS_VACUUM ring buffer should be
BLCKSZ * pinned_buffer_multiplier * default maintenance_io_concurrency
(probably rounded up to the next power of two) bytes.

- Melanie

Commits

  1. Fix explicit valgrind interaction in read_stream.c.

  2. Reduce scope of heap vacuum per_buffer_data

  3. Use streaming read I/O in VACUUM's third phase

  4. Use streaming read I/O in VACUUM's first phase

  5. Convert heap_vac_scan_next_block() boolean parameters to flags

  6. Refactor tidstore.c iterator buffering.

  7. Increase default vacuum_buffer_usage_limit to 2MB.

  8. Remove unneeded vacuum_delay_point from heap_vac_scan_get_next_block

  9. Confine vacuum skip logic to lazy_scan_skip()

  10. Set all_visible_according_to_vm correctly with DISABLE_PAGE_SKIPPING

  11. Tighten up VACUUM's approach to setting VM bits.