Horton

Re: Warn when parallel restoring a custom dump without data offsets

Justin Pryzby <pryzby@telsasoft.com>

View thread View on postgr.es
From: Justin Pryzby <pryzby@telsasoft.com>
To: David Gilman <davidgilman1@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2020-05-20T03:26:57Z
Lists: pgsql-hackers 
I started fooling with this at home while our ISP is broke (pardon my brevity).

Maybe you also saw commit b779ea8a9a2dc3a089b3ac152b1ec4568bfeb26f
"Fix pg_restore so parallel restore doesn't fail when the input file
doesn't contain data offsets (which it won't, if pg_dump thought its
output wasn't seekable)..."

...which I guess should actually say "doesn't NECESSARILY fail", since
it also adds this comment:
"This could fail if we are asked to restore items out-of-order."

So this is a known issue and not a regression.  I think the PG11
commit you mentioned (548e5097) happens to make some databases fail in
parallel restore that previously worked (I didn't check).  Possibly
also some databases (or some pre-existing dumps) which used to fail
might possibly now succeed.

Your patch adds a warning if unseekable output might fail during
parallel restore.  I'm not opposed to that, but can we just make
pg_restore work in that case?  If the input is unseekable, then we can
never do a parallel restore at all.  If it *is* seekable, could we
make _PrintTocData rewind if it gets to EOF using ftello(SEEK_SET, 0)
and re-scan again from the beginning?  Would you want to try that ?

Commits

  1. Cope with data-offset-less archive files during out-of-order restores.

  2. Remove manual tracking of file position in pg_dump/pg_backup_custom.c.

  3. Improve performance of tuple conversion map generation

  4. Fix pg_restore so parallel restore doesn't fail when the input file doesn't