pg_upgrade --copy-file-range
Thomas Munro <thomas.munro@gmail.com>
From: Thomas Munro <thomas.munro@gmail.com>
To: pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2023-06-02T19:30:44Z
Lists: pgsql-hackers
Attachments
- 0001-Add-copy-file-range-option-to-pg_upgrade.patch (text/x-patch) patch 0001
Hello, I was just in a pg_upgrade unconference session at PGCon where the lack of $SUBJECT came up. This system call gives the kernel the option to use fast block cloning on XFS, ZFS (as of very recently), etc, and works on Linux and FreeBSD. It's probably much the same as --clone mode on COW file systems, except that is Linux-only. On overwrite file systems (ie not copy-on-write, like ext4), it may also be able to push copies down to storage hardware/network file systems. There was something like this in the nearby large files patch set, but in that version it just magically did it when available in --copy mode. Now I think the user should have to have to opt in with --copy-file-range, and simply to error out if it fails. It may not work in some cases -- for example, the man page says that older Linux systems can fail with EXDEV when you try to copy across file systems, while newer systems will do something less efficient but still sensible internally; also I saw a claim that some older versions had weird bugs. Better to just expose the raw functionality and let users say when they want it and read the error if it fail, I think.
Commits
-
Allow using copy_file_range in write_reconstructed_file
- ac8110155132 17.0 landed
-
Allow copying files using clone/copy_file_range
- f8ce4ed78ca6 17.0 landed
-
Align blocks in incremental backups to BLCKSZ
- 10e3226ba13d 17.0 landed
-
Add --copy-file-range option to pg_upgrade.
- d93627bcbe50 17.0 landed