Re: optimize file transfer in pg_upgrade

Greg Sabino Mullane <htamfids@gmail.com>

From: Greg Sabino Mullane <htamfids@gmail.com>
To: Nathan Bossart <nathandbossart@gmail.com>
Cc: Robert Haas <robertmhaas@gmail.com>, pgsql-hackers@postgresql.org
Date: 2025-03-05T20:40:52Z
Lists: pgsql-hackers
On Wed, Mar 5, 2025 at 2:43 PM Nathan Bossart <nathandbossart@gmail.com>
wrote:

> One other design point I wanted to bring up is whether we should bother
> generating a rollback script for the new "swap" mode.  In short, I'm
> wondering if it would be unreasonable to say that, just for this mode, once
> pg_upgrade enters the file transfer step, reverting to the old cluster
> requires restoring a backup.


I think that's a fair requirement. And like Robert, revert scripts make me
nervous.

* Anecdotally, I'm not sure I've ever actually seen pg_upgrade fail
> during or after file transfer, and I'm hoping to get some real data about
> that in the near future.  Has anyone else dealt with such a failure?


I've seen various failures, but they always get caught quite early.
Certainly early enough to easily abort, fix perms/mounts/etc., then retry.
I think your instinct is correct that this reversion is more trouble than
its worth. I don't think the pg_upgrade docs mention taking a backup, but
that's always step 0 in my playbook, and that's the rollback plan in the
unlikely event of failure.

Cheers,
Greg

--
Crunchy Data - https://www.crunchydata.com
Enterprise Postgres Software Products & Tech Support

Commits

  1. initdb: Add --no-sync-data-files.

  2. pg_dump: Add --sequence-data.

  3. pg_upgrade: Add --swap for faster file transfer.

  4. Add test for pg_upgrade file transfer modes.

  5. Fix an intermetant BF failure in 003_logical_slots.