Re: Inefficiency in parallel pg_restore with many tables
Robert Haas <robertmhaas@gmail.com>
From: Robert Haas <robertmhaas@gmail.com>
To: Nathan Bossart <nathandbossart@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Andrew Dunstan <andrew@dunslane.net>, pgsql-hackers@lists.postgresql.org
Date: 2023-09-01T20:00:44Z
Lists: pgsql-hackers
On Tue, Jul 25, 2023 at 2:53 PM Nathan Bossart <nathandbossart@gmail.com> wrote: > On Mon, Jul 24, 2023 at 12:00:15PM -0700, Nathan Bossart wrote: > > Here is a sketch of this approach. It required fewer #ifdefs than I was > > expecting. At the moment, this one seems like the winner to me. > > Here is a polished patch set for this approach. I've also added a 0004 > that replaces the open-coded heap in pg_dump_sort.c with a binaryheap. > IMHO these patches are in decent shape. [ drive-by comment that hopefully doesn't cause too much pain ] In hindsight, I think that making binaryheap depend on Datum was a bad idea. I think that was my idea, and I think it wasn't very smart. Considering that people have coded to that decision up until now, it might not be too easy to change at this point. But in principle I guess you'd want to be able to make a heap out of any C data type, rather than just Datum, or just Datum in the backend and just void * in the frontend. -- Robert Haas EDB: http://www.enterprisedb.com
Commits
-
Remove open-coded binary heap in pg_dump_sort.c.
- 559bc1732180 17.0 landed
-
Convert pg_restore's ready_list to a priority queue.
- 9bfd44bbde42 17.0 landed
-
Add function for removing arbitrary nodes in binaryheap.
- c103d073819a 17.0 landed
-
Make binaryheap available to frontend code.
- 5af0263afd7b 17.0 landed