Re: Horribly slow pg_upgrade performance with many Large Objects

Nathan Bossart <nathandbossart@gmail.com>

From: Nathan Bossart <nathandbossart@gmail.com>

To: Hannu Krosing <hannuk@google.com>

Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>

Date: 2025-04-07T22:17:03Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

pg_upgrade: Use COPY for large object metadata.
- 161a3e8b682e 19 (unreleased) landed
pg_dump: Fix object-type sort priority for large objects.
- fb6c860bbd1f 19 (unreleased) landed
- e43fae199e50 17.6 landed
- 36026b0fe3ab 18.0 landed
pg_upgrade: Mention that we preserve database OIDs in a comment.
- e2665efd0f4d 18.0 landed
pg_upgrade: Preserve database OIDs.
- aa01051418f1 15.0 cited
Fix pg_upgrade for oid removal.
- 12a53c732ced 12.0 cited

On Mon, Apr 07, 2025 at 10:33:47PM +0200, Hannu Krosing wrote:
> The obvious solution would be to handle the table
> `pg_largeobject_metadata` the same way as we currently handle
> `pg_largeobject `by not doing anything with it in `pg_dump
> --binary-upgrade` and just handle the contents it like we do for user
> tables in pg_upgrade itself.
> 
> This should work fine for all source database versions starting from PgSQL v12.

Unfortunately, the storage format for aclitem changed in v16, so this would
need to be restricted to upgrades from v16 and newer.  That being said, I
regularly hear about slow upgrades with many LOs, so I think it'd be
worthwhile to try to improve matters in v19.

-- 
nathan