Re: pg_dump versus hash partitioning
Tom Lane <tgl@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Julien Rouhaud <rjuju123@gmail.com>
Cc: Andrew Dunstan <andrew@dunslane.net>,
Alvaro Herrera <alvherre@alvh.no-ip.org>,
David Rowley <dgrowleyml@gmail.com>, Peter Geoghegan <pg@bowt.ie>,
Robert Haas <robertmhaas@gmail.com>,
pgsql-hackers@lists.postgresql.org,
Andrew <pgsqlhackers@andrewrepp.com>
Date: 2023-03-13T23:39:12Z
Lists: pgsql-hackers
Attachments
- v2-0001-Create-a-test-case-for-dump-and-restore-of-hashed.patch (text/x-diff) patch v2-0001
- v2-0002-Avoid-TRUNCATE-when-restoring-load-via-partition-.patch (text/x-diff) patch v2-0002
- v2-0003-Add-a-marker-to-TABLE-DATA-items-using-load-via-p.patch (text/x-diff) patch v2-0003
- v2-0004-Force-load-via-partition-root-for-hash-partitioni.patch (text/x-diff) patch v2-0004
- v2-0005-Don-t-create-useless-TableAttachInfo-objects.patch (text/x-diff) patch v2-0005
- v2-0006-Simplify-pg_dump-s-creation-of-parent-table-links.patch (text/x-diff) patch v2-0006
Julien Rouhaud <rjuju123@gmail.com> writes: > On Sun, Mar 12, 2023 at 03:46:52PM -0400, Tom Lane wrote: >> The trick is to detect in pg_restore whether pg_dump chose to do >> load-via-partition-root. > Given that this approach wouldn't help with existing dump files (at least if > using COPY, in any case the one using INSERT are doomed), so I'm slightly in > favor of the first approach, and later add an easy and non magic incantation > way to produce dumps that don't depend on partitioning. Yeah, we need to do both. Attached find an updated patch series: 0001: TAP test that exhibits both this deadlock problem and the different-hash-codes problem. I'm not sure if we want to commit this, or if it should be in exactly this form --- the second set of tests with a manual --load-via-partition-root switch will be pretty redundant after this patch series. 0002: Make pg_restore detect load-via-partition-root by examining the COPY commands embedded in the dump, and skip the TRUNCATE if so, thereby fixing the deadlock issue. This is the best we can do for legacy dump files, I think, but it should be good enough. 0003: Also detect load-via-partition-root by adding a label in the dump. This is a more bulletproof solution going forward. 0004-0006: same as previous patches, but rebased over these. This gets us to a place where the new TAP test passes. I've not done anything about modifying the documentation, but I still think we could remove the warning label on --load-via-partition-root. regards, tom lane
Commits
-
Simplify and speed up pg_dump's creation of parent-table links.
- 064709f803c0 16.0 landed
-
Fix pg_dump for hash partitioning on enum columns.
- bc8cd50fefd3 16.0 landed
- 8f83ce8c5244 12.15 landed
- 7e7c5b683985 13.11 landed
- 5fc1ac151d85 14.8 landed
- 2b216da1e55d 15.3 landed
- 012ffb365a05 11.20 landed