Convert pg_restore's ready_list to a priority queue.
Nathan Bossart <nathan@postgresql.org>
Convert pg_restore's ready_list to a priority queue. Presently, parallel restores spend a lot of time sorting this list so that we pick the largest items first. With many tables, this sorting can become a significant bottleneck. There are a couple of reports from the field about this, and it is easy to reproduce. This commit improves the performance of parallel pg_restore with many tables by converting its ready_list to a priority queue, i.e., a binary heap. We will first try to run the highest priority item, but if it cannot be chosen due to the lock heuristic, we'll do a sequential scan through the heap nodes until we find one that is runnable. This means that we might end up picking an item with a much lower priority. However, we expect that we will typically be able to pick one of the first few items, which should usually have a relatively high priority. Suggested-by: Tom Lane Tested-by: Pierre Ducroquet Reviewed-by: Tom Lane Discussion: https://postgr.es/m/3612876.1689443232%40sss.pgh.pa.us
Files
| Path | Change | +/− |
|---|---|---|
| src/bin/pg_dump/pg_backup_archiver.c | modified | +58 −140 |
Discussion
- Inefficiency in parallel pg_restore with many tables 37 messages · 2023-07-15 → 2023-09-20