Horton

Re: Warn when parallel restoring a custom dump without data offsets

Tom Lane <tgl@sss.pgh.pa.us>

View thread View on postgr.es
From: Tom Lane <tgl@sss.pgh.pa.us>
To: David Gilman <davidgilman1@gmail.com>
Cc: Justin Pryzby <pryzby@telsasoft.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2020-05-20T14:48:30Z
Lists: pgsql-hackers 
David Gilman <davidgilman1@gmail.com> writes:
>> I think the PG11
>> commit you mentioned (548e5097) happens to make some databases fail in
>> parallel restore that previously worked (I didn't check).

> Correct, if you do the bisect around that yourself you'll see
> pg_restore start failing with the expected "possibly due to
> out-of-order restore request" on offset-less dumps.

Yeah.  Now, the whole point of that patch was to decouple the restore
order from the dump order ... but with an offset-less dump file, we
can't do that, or at least the restore order is greatly constrained.
I wonder if it'd be sensible for pg_restore to use a different parallel
scheduling algorithm if it notices that the input lacks offsets.
(There could still be some benefit from parallelism, just not as much.)
No idea if this is going to be worth the trouble, but it probably
is worth looking into.

			regards, tom lane

Commits

  1. Cope with data-offset-less archive files during out-of-order restores.

  2. Remove manual tracking of file position in pg_dump/pg_backup_custom.c.

  3. Improve performance of tuple conversion map generation

  4. Fix pg_restore so parallel restore doesn't fail when the input file doesn't