Horton
Menu

Re: pg_upgrade failing for 200+ million Large Objects

Tom Lane <tgl@sss.pgh.pa.us>

View thread
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Alexander Korotkov <aekorotkov@gmail.com>
Cc: Justin Pryzby <pryzby@telsasoft.com>, Nathan Bossart <nathandbossart@gmail.com>, Michael Banck <mbanck@gmx.net>, Laurenz Albe <laurenz.albe@cybertec.at>, vignesh C <vignesh21@gmail.com>, "Kumar, Sachin" <ssetiya@amazon.com>, Robins Tharakan <tharakan@gmail.com>, Jan Wieck <jan@wi3ck.info>, Bruce Momjian <bruce@momjian.us>, Andrew Dunstan <andrew@dunslane.net>, Magnus Hagander <magnus@hagander.net>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, pgsql-hackers@postgresql.org
Date: 2024-07-26T20:05:50Z
Lists: pgsql-hackers 
Alexander Korotkov <aekorotkov@gmail.com> writes:
> On Wed, Jul 24, 2024 at 5:18 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
>> We'd surely prefer that the transaction size be configurable.

> I think we can add an option to pg_upgrade.  But I wonder if there is
> something else we can do.

Yeah, I'm not enamored of adding a command-line option, if only
because I think a lot of people invoke pg_upgrade through
vendor-provided scripts that aren't going to cooperate with that.
If we can find some way to make it adapt without help, that
would be much better.

			regards, tom lane

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Count individual SQL commands in pg_restore's --transaction-size mode.

  2. Reduce number of commands dumpTableSchema emits for binary upgrade.

  3. Invent --transaction-size option for pg_restore.

  4. Rearrange pg_dump's handling of large objects for better efficiency.

  5. Add temporal PRIMARY KEY and UNIQUE constraints

  6. Fix typo and case in messages