Re: Horribly slow pg_upgrade performance with many Large Objects
Tom Lane <tgl@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Nathan Bossart <nathandbossart@gmail.com>
Cc: Hannu Krosing <hannuk@google.com>,
PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2025-04-08T17:07:09Z
Lists: pgsql-hackers
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
pg_upgrade: Use COPY for large object metadata.
- 161a3e8b682e 19 (unreleased) landed
-
pg_dump: Fix object-type sort priority for large objects.
- fb6c860bbd1f 19 (unreleased) landed
- e43fae199e50 17.6 landed
- 36026b0fe3ab 18.0 landed
-
pg_upgrade: Mention that we preserve database OIDs in a comment.
- e2665efd0f4d 18.0 landed
-
pg_upgrade: Preserve database OIDs.
- aa01051418f1 15.0 cited
-
Fix pg_upgrade for oid removal.
- 12a53c732ced 12.0 cited
Nathan Bossart <nathandbossart@gmail.com> writes:
> I do think it's worth considering going back to copying
> pg_largobject_metadata's files for upgrades from v16 and newer.
(If we do this) I don't see why we'd need to stop at v16. I'm
envisioning that we'd use COPY, which will be dealing in the
text representation of aclitems, and I don't think that's changed
in a long time. The sort of thing that would break it is changes
in the set of available/default privilege bits for large objects.
That is, where the dump currently contains something like
SELECT pg_catalog.lo_create('2121');
ALTER LARGE OBJECT 2121 OWNER TO postgres;
GRANT ALL ON LARGE OBJECT 2121 TO joe;
we'd have
COPY pg_largeobject_metadata FROM STDIN;
...
2121 10 {postgres=rw/postgres,joe=rw/postgres}
...
and some appropriate COPY data for pg_shdepend too.
(The fact that this representation will contain both numeric and
symbolic role OIDs is why I was concerned about OID stability.)
The key thing that worries me here is if the source and target
versions have different ideas of which roles are pinned, which would
silently change what appears in pg_shdepend. But it'd only really
break if a role mentioned in some LO's owner or ACL is pinned in the
source and not in the target, which seems unlikely. (In the other
direction, we'd just be adding a useless row in pg_shdepend.)
regards, tom lane