Re: Inefficiency in parallel pg_restore with many tables

Alvaro Herrera <alvherre@alvh.no-ip.org>

From: Alvaro Herrera <alvherre@alvh.no-ip.org>
To: Nathan Bossart <nathandbossart@gmail.com>
Cc: Robert Haas <robertmhaas@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, Andrew Dunstan <andrew@dunslane.net>, pgsql-hackers@lists.postgresql.org
Date: 2023-09-03T10:04:00Z
Lists: pgsql-hackers
On 2023-Sep-02, Nathan Bossart wrote:

> On Fri, Sep 01, 2023 at 01:52:48PM -0700, Nathan Bossart wrote:

> > Yeah, something similar to simplehash for binary heaps could be nice.  That
> > being said, I don't know if there's a strong reason to specialize the
> > implementation for a given C data type in most cases.
> 
> I ended up hacking together a (nowhere near committable) patch to see how
> hard it would be to allow using any type with binaryheap.  It doesn't seem
> too bad.

Yeah, using void * seems to lead to interfaces that are pretty much the
same as bsearch() or qsort().  (Why isn't your payload type const,
though?)

I do wonder why did you change _remove_first and _first to have a
'result' output argument instead of a return value.  Does this change
actually buy you anything?  simplehash.h doesn't do that either.

> -extern void binaryheap_add(binaryheap *heap, Datum d);
> -extern Datum binaryheap_first(binaryheap *heap);
> -extern Datum binaryheap_remove_first(binaryheap *heap);
> -extern void binaryheap_replace_first(binaryheap *heap, Datum d);
> +extern void binaryheap_add(binaryheap *heap, void *d);
> +extern void binaryheap_first(binaryheap *heap, void *result);
> +extern void binaryheap_remove_first(binaryheap *heap, void *result);
> +extern void binaryheap_replace_first(binaryheap *heap, void *d);

-- 
Álvaro Herrera        Breisgau, Deutschland  —  https://www.EnterpriseDB.com/



Commits

  1. Remove open-coded binary heap in pg_dump_sort.c.

  2. Convert pg_restore's ready_list to a priority queue.

  3. Add function for removing arbitrary nodes in binaryheap.

  4. Make binaryheap available to frontend code.