Re: Adding pg_dump flag for parallel export to pipes
Nitin Motiani <nitinmotiani@google.com>
From: Nitin Motiani <nitinmotiani@google.com>
To: Hannu Krosing <hannuk@google.com>
Cc: Mahendra Singh Thalor <mahi6run@gmail.com>,
Dilip Kumar <dilipbalaut@gmail.com>, Thomas Munro <thomas.munro@gmail.com>, pgsql-hackers@postgresql.org
Date: 2026-05-04T04:39:13Z
Lists: pgsql-hackers
Attachments
- v11-0003-Fixes-and-refactors-in-pipe-command.patch (application/x-patch) patch v11-0003
- v11-0004-Add-tests-for-pipe.patch (application/x-patch) patch v11-0004
- v11-0005-Add-documentation-for-pipe-in-pg_dump-and-pg_res.patch (application/x-patch) patch v11-0005
- v11-0002-Add-pipe-command-support-in-pg_restore.patch (application/x-patch) patch v11-0002
- v11-0001-Add-pipe-command-support-for-directory-mode-of-p.patch (application/x-patch) patch v11-0001
Hi Mahendra, I'm attaching the latest version of the patch. This incorporates several suggestions Mahendra made. I've also fixed the failing test and added support for Windows in the test by using perlbin instead of cat. Here is the list of changes in this version: 1. Changed pipe-command to pip. 2. Added better error handling for invalid command. 3. Added shell escaping in the command before setting it as the file path. 4. I decided to change the mode to PG_BINARY_W for the large object toc files even in the standard case. If there is a concern, we can revert to the older version. 5. Added a bunch of new tests for various scenarios and parallelism. Currently these changes are in a separate commit. After the review, I can squash the first 3 commits of pg_dump, pg_restore, and the fixes in one commit. On Sat, Mar 14, 2026 at 10:37 PM Mahendra Singh Thalor <mahi6run@gmail.com> wrote: > > Comment5: I think we can support this new pipe option with pg_dumpall also as we support directory mode in pg_dumpall from v19. Regarding this, I haven't looked into the details of how that would work. If the rest look good, we can consider implementing this in another patch. For the time being, I've skipped this option in pg_dumpall and the global restore. Let me know what you think. Thanks Nitin Motiani, Google