Re: Inefficiency in parallel pg_restore with many tables
Nathan Bossart <nathandbossart@gmail.com>
From: Nathan Bossart <nathandbossart@gmail.com>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: Robert Haas <robertmhaas@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, Andrew Dunstan <andrew@dunslane.net>, pgsql-hackers@lists.postgresql.org
Date: 2023-09-03T15:11:16Z
Lists: pgsql-hackers
On Sun, Sep 03, 2023 at 12:04:00PM +0200, Alvaro Herrera wrote: > On 2023-Sep-02, Nathan Bossart wrote: >> I ended up hacking together a (nowhere near committable) patch to see how >> hard it would be to allow using any type with binaryheap. It doesn't seem >> too bad. > > Yeah, using void * seems to lead to interfaces that are pretty much the > same as bsearch() or qsort(). Right. This is what I had in mind. > (Why isn't your payload type const, > though?) It probably should be const. This patch was just a proof-of-concept and still requireѕ a bit of work. > I do wonder why did you change _remove_first and _first to have a > 'result' output argument instead of a return value. Does this change > actually buy you anything? simplehash.h doesn't do that either. > >> -extern void binaryheap_add(binaryheap *heap, Datum d); >> -extern Datum binaryheap_first(binaryheap *heap); >> -extern Datum binaryheap_remove_first(binaryheap *heap); >> -extern void binaryheap_replace_first(binaryheap *heap, Datum d); >> +extern void binaryheap_add(binaryheap *heap, void *d); >> +extern void binaryheap_first(binaryheap *heap, void *result); >> +extern void binaryheap_remove_first(binaryheap *heap, void *result); >> +extern void binaryheap_replace_first(binaryheap *heap, void *d); _first could likely just return a pointer to the data in the binary heap's array. However, _remove_first has to copy the data somewhere, so I think the alternative would be to return a palloc'd value. Is there another way that I'm not thinking of? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com
Commits
-
Remove open-coded binary heap in pg_dump_sort.c.
- 559bc1732180 17.0 landed
-
Convert pg_restore's ready_list to a priority queue.
- 9bfd44bbde42 17.0 landed
-
Add function for removing arbitrary nodes in binaryheap.
- c103d073819a 17.0 landed
-
Make binaryheap available to frontend code.
- 5af0263afd7b 17.0 landed