Horton

Re: Warn when parallel restoring a custom dump without data offsets

David Gilman <dgilman@gilslotd.com>

View thread View on postgr.es
From: David Gilman <dgilman@gilslotd.com>
To: Justin Pryzby <pryzby@telsasoft.com>
Cc: David Gilman <davidgilman1@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, pgsql-hackers@lists.postgresql.org, thomas.munro@gmail.com
Date: 2020-05-25T21:55:26Z
Lists: pgsql-hackers

Attachments

The earlier patches weren't applying because I had "git config
diff.noprefix true" set globally and that was messing up the git
format-patch output.

On Mon, May 25, 2020 at 01:54:29PM -0500, David Gilman wrote:
> And I misunderstood how bad it was. I thought it was reading little
> header structs off the disk but it's actually reading the entire table
> (see _skipData). So you're quadratically rereading entire tables and
> thrashing your cache. Oops.

I changed _skipData to fseeko() instead of fread() when possible to cut
down on this thrashing further.

-- 
David Gilman  :DG<
https://gilslotd.com

Commits

  1. Cope with data-offset-less archive files during out-of-order restores.

  2. Remove manual tracking of file position in pg_dump/pg_backup_custom.c.

  3. Improve performance of tuple conversion map generation

  4. Fix pg_restore so parallel restore doesn't fail when the input file doesn't