Re: Warn when parallel restoring a custom dump without data offsets
Justin Pryzby <pryzby@telsasoft.com>
From: Justin Pryzby <pryzby@telsasoft.com>
To: David Gilman <dgilman@gilslotd.com>
Cc: David Gilman <davidgilman1@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, pgsql-hackers@lists.postgresql.org, thomas.munro@gmail.com
Date: 2020-06-13T22:51:21Z
Lists: pgsql-hackers
On Mon, May 25, 2020 at 01:54:29PM -0500, David Gilman wrote:
> > Is it possible to dump to stdout (or pipe to cat or dd) to avoid a new option ?
>
> The underlying IPC::Run code seems to support piping in a cross-platform
> way. I am not a Perl master though and after spending an evening trying
> to get it to work I went with this approach. If you can put me in touch
> with anyone to help me out here I'd appreciate it.
I think you can do what's needed like so:
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -152,10 +152,13 @@ my %pgdump_runs = (
},
defaults_custom_format_no_seek_parallel_restore => {
test_key => 'defaults',
- dump_cmd => [
- 'pg_dump', '-Fc', '-Z6', '--no-sync', '--disable-seeking',
+ dump_cmd => (
+ [
+ 'pg_dump', '-Fc', '-Z6', '--no-sync',
"--file=$tempdir/defaults_custom_format_no_seek_parallel_restore.dump", 'postgres',
- ],
+ ],
+ "|", [ "cat" ], # disable seeking
+ ),
Also, these are failing intermittently:
t/002_pg_dump.pl .............. 1649/6758
# Failed test 'defaults_custom_format_no_seek_parallel_restore: should dump GRANT SELECT (proname ...) ON TABLE pg_proc TO public'
# at t/002_pg_dump.pl line 3635.
# Review defaults_custom_format_no_seek_parallel_restore results in /var/lib/pgsql/postgresql.src/src/bin/pg_dump/tmp_check/tmp_test_NqRC
t/002_pg_dump.pl .............. 2060/6758
# Failed test 'defaults_dir_format_parallel: should dump GRANT SELECT (proname ...) ON TABLE pg_proc TO public'
# at t/002_pg_dump.pl line 3635.
# Review defaults_dir_format_parallel results in /var/lib/pgsql/postgresql.src/src/bin/pg_dump/tmp_check/tmp_test_NqRC
If you can address those, I think this will be "ready for committer".
--
Justin
Commits
-
Cope with data-offset-less archive files during out-of-order restores.
- 71e8e66f783c 13.0 landed
- f009591d6edd 14.0 landed
- 66232220ee6e 12.4 landed
-
Remove manual tracking of file position in pg_dump/pg_backup_custom.c.
- 447cf2f8e9dc 13.0 landed
- a8d0732ac2b5 14.0 landed
- 39a068ce6675 12.4 landed
-
Improve performance of tuple conversion map generation
- 42f70cd9c3db 12.0 cited
-
Fix pg_restore so parallel restore doesn't fail when the input file doesn't
- b779ea8a9a2d 9.0.0 cited