Re: pg_upgrade --copy-file-range

Thomas Munro <thomas.munro@gmail.com>

From: Thomas Munro <thomas.munro@gmail.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Tomas Vondra <tomas.vondra@enterprisedb.com>, Jakub Wartak <jakub.wartak@enterprisedb.com>, Peter Eisentraut <peter@eisentraut.org>, Michael Paquier <michael@paquier.xyz>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2024-03-23T00:25:38Z
Lists: pgsql-hackers
Hmm, this discussion seems to assume that we only use
copy_file_range() to copy/clone whole segment files, right?  That's
great and may even get most of the available benefit given typical
databases with many segments of old data that never changes, but... I
think copy_write_range() allows us to go further than the other
whole-file clone techniques: we can stitch together parts of an old
backup segment file and an incremental backup to create a new file.
If you're interested in minimising disk use while also removing
dependencies on the preceding chain of backups, then it might make
sense to do that even if you *also* have to read the data to compute
the checksums, I think?  That's why I mentioned it: if
copy_file_range() (ie sub-file-level block sharing) is a solution in
search of a problem, has the world ever seen a better problem than
pg_combinebackup?



Commits

  1. Allow using copy_file_range in write_reconstructed_file

  2. Allow copying files using clone/copy_file_range

  3. Align blocks in incremental backups to BLCKSZ

  4. Add --copy-file-range option to pg_upgrade.