Horton
Menu

Re: pg_upgrade failing for 200+ million Large Objects

Tom Lane <tgl@sss.pgh.pa.us>

View thread
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Alexander Korotkov <aekorotkov@gmail.com>
Cc: Justin Pryzby <pryzby@telsasoft.com>, Nathan Bossart <nathandbossart@gmail.com>, Michael Banck <mbanck@gmx.net>, Laurenz Albe <laurenz.albe@cybertec.at>, vignesh C <vignesh21@gmail.com>, "Kumar, Sachin" <ssetiya@amazon.com>, Robins Tharakan <tharakan@gmail.com>, Jan Wieck <jan@wi3ck.info>, Bruce Momjian <bruce@momjian.us>, Andrew Dunstan <andrew@dunslane.net>, Magnus Hagander <magnus@hagander.net>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, pgsql-hackers@postgresql.org
Date: 2024-07-27T03:08:06Z
Lists: pgsql-hackers 
Alexander Korotkov <aekorotkov@gmail.com> writes:
> J4F, I have an idea to count number of ';' sings and use it for
> transaction size counter, since it is as upper bound estimate of
> number of SQL commands :-)

Hmm ... that's not a completely silly idea.  Let's keep it in
the back pocket in case we can't easily reduce the number of
SQL commands in some cases.

It's late here, and I've got some other commitments tomorrow,
but I'll try to produce a patch to merge more of the SQL
commands in a day or two.

			regards, tom lane

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Count individual SQL commands in pg_restore's --transaction-size mode.

  2. Reduce number of commands dumpTableSchema emits for binary upgrade.

  3. Invent --transaction-size option for pg_restore.

  4. Rearrange pg_dump's handling of large objects for better efficiency.

  5. Add temporal PRIMARY KEY and UNIQUE constraints

  6. Fix typo and case in messages