Horton

Re: Inefficiency in parallel pg_restore with many tables

Tom Lane <tgl@sss.pgh.pa.us>

View thread View on postgr.es
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Nathan Bossart <nathandbossart@gmail.com>
Cc: Robert Haas <robertmhaas@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Andrew Dunstan <andrew@dunslane.net>, pgsql-hackers@lists.postgresql.org
Date: 2023-09-14T00:01:39Z
Lists: pgsql-hackers 
Nathan Bossart <nathandbossart@gmail.com> writes:
> Upon closer inspection, I found a rather nasty problem.  The qsort
> comparator expects a TocEntry **, but the binaryheap comparator expects a
> TocEntry *, and we simply pass the arguments through to the qsort
> comparator.  In v9, I added the requisite ampersands.

Ooops :-(

> I'm surprised this
> worked at all.

Probably it was not sorting things appropriately.  Might be worth adding
some test scaffolding to check that bigger tasks are chosen before
smaller ones.

			regards, tom lane

Commits

  1. Remove open-coded binary heap in pg_dump_sort.c.

  2. Convert pg_restore's ready_list to a priority queue.

  3. Add function for removing arbitrary nodes in binaryheap.

  4. Make binaryheap available to frontend code.