Re: pg_dump test instability
Tom Lane <tgl@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Cc: Stephen Frost <sfrost@snowman.net>,
pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2018-09-13T21:03:39Z
Lists: pgsql-hackers
Attachments
- smarter-parallel-dump-restore-2.patch (text/x-diff) patch
I wrote: > Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes: >> I see. Why not shift all items up to the i'th up by one place, instead >> of moving only the first one? That way the sortedness would be >> preserved. Otherwise we'd move the first one into the middle, then >> sorting would move it to the front again, etc. > Hmmm ... might be worth doing, but I'm not sure. The steady-state cycle > will probably be that after one task has been dispatched, we'll sleep > until some task finishes and then that will unblock some pending items, > resulting in new entries at the end of the list, forcing a sort anyway > before we next dispatch a task. So I was expecting that avoiding a sort > here wasn't really going to be worth expending much effort for. But my > intuition about that could be wrong. I'll run a test case with some > instrumentation added and see how often we could avoid sorts by > memmove'ing. OK, my intuition was faulty. At least when testing with the regression database, situations where we are taking the slow path at all seem to involve several interrelated dump objects (eg indexes of a table) that are all waiting for the same lock, such that we may be able to dispatch a number of unrelated tasks before anything gets added from the pending list. Doing it as you suggest eliminates a significant fraction of the re-sort operations. Attached updated patch does it like that and makes the cosmetic adjustments you suggested. I also went ahead and did the renaming of par_prev/par_next/par_list_xxx that I'd suggested upthread. I think this is committable ... regards, tom lane
Commits
-
Improve parallel scheduling logic in pg_dump/pg_restore.
- 548e50976ce7 12.0 landed