Re: optimize file transfer in pg_upgrade

Alvaro Herrera <alvherre@alvh.no-ip.org>

From: Álvaro Herrera <alvherre@alvh.no-ip.org>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Nathan Bossart <nathandbossart@gmail.com>, Greg Sabino Mullane <htamfids@gmail.com>, pgsql-hackers@postgresql.org, bruce@momjian.us
Date: 2025-03-18T15:51:43Z
Lists: pgsql-hackers
On 2025-Mar-18, Robert Haas wrote:

> The background here is that I'm kind of on the warpath against weird
> configurations that we only test on certain buildfarm animals at the
> moment, because the result of that is that CI is clean and then the
> buildfarm turns red when you commit. That's an unenjoyable experience
> for the committer and for everyone who looks at the buildfarm results.
> The way to fix it is to stop relying on "rerun all the tests with this
> weird mode flag" and rely more on tests that are designed to test that
> specific flag and, ideally, that get run by in local testing or at
> least by CI.

FWIW this is exactly the rationale that got me writing an email on the
Ashutosh's thread for a new pg_dump/restore test under
002_pg_upgrade.pl, whereby I was saying that we should not hide it
behind PG_TEST_EXTRA which almost nobody would remember to use.  But I
discarded that draft, because that had actually been Ashutosh's idea at
some point in the thread and had been discarded because of the runtime
increase it'd cause.  But, somehow, I still don't believe the theory
that it's such a bad idea to add a few seconds so that we have such a
comprehensive pg_dump test, with much less programmer overhead than
pg_dump's own weird enormous test script.

-- 
Álvaro Herrera               48°01'N 7°57'E  —  https://www.EnterpriseDB.com/
"Cada quien es cada cual y baja las escaleras como quiere" (JMSerrat)



Commits

  1. initdb: Add --no-sync-data-files.

  2. pg_dump: Add --sequence-data.

  3. pg_upgrade: Add --swap for faster file transfer.

  4. Add test for pg_upgrade file transfer modes.

  5. Fix an intermetant BF failure in 003_logical_slots.