v13-0005-Add-documentation-for-pipe-in-pg_dump-and-pg_res.patch

From 6ba7e3834c56b27aad1e109f0051b1ef4fd77323 Mon Sep 17 00:00:00 2001
From: Nitin Motiani <nitinmotiani@google.com>
Date: Fri, 4 Apr 2025 14:34:48 +0000
Subject: [PATCH v13 5/5] Add documentation for pipe in pg_dump and pg_restore

* Add the descriptions of the new flags and constraints
  regarding which mode and other flags they can't be used with.
* Explain the purpose of the flags.
* Add a few examples of the usage of the flags.
---
 doc/src/sgml/ref/pg_dump.sgml    | 56 ++++++++++++++++++++++++++
 doc/src/sgml/ref/pg_restore.sgml | 68 +++++++++++++++++++++++++++++++-
 2 files changed, 123 insertions(+), 1 deletion(-)

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index ae1bc14d2f2..6458b032d25 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -297,6 +297,7 @@ PostgreSQL documentation
         specifies the target directory instead of a file. In this case the
         directory is created by <command>pg_dump</command> unless the directory
         exists and is empty.
+        This option and <option>--pipe</option> can't be used together.
        </para>
       </listitem>
      </varlistentry>
@@ -1224,6 +1225,32 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--pipe</option></term>
+      <listitem>
+       <para>
+        This option is only supported with the directory output
+        format. It can be used to write to multiple streams which
+        otherwise would not be possible with the directory mode.
+        For each stream, it starts a process which runs the
+        specified command and pipes the pg_dump output to this
+        process.
+        This option is not valid if <option>--file</option>
+        is also specified.
+       </para>
+       <para>
+        The pipe can be used to perform operations like compress
+        using a custom algorithm, filter, or write the output to a cloud
+        storage etc. The user would need a way to pipe the final output of
+        each stream to a file. To handle that, the pipe command supports a format
+        specifier %f. And all the instances of %f in the command string
+        will be replaced with the corresponding file name which
+        would have been used in the directory mode with <option>--file</option>.
+        See <xref linkend="pg-dump-examples"/> below.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--quote-all-identifiers</option></term>
       <listitem>
@@ -1803,6 +1830,35 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 </screen>
   </para>
 
+  <para>
+   To use pipe to dump a database into a directory-format archive
+   (the directory <literal>dumpdir</literal> needs to exist before running the command).
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump -Fd mydb --pipe="cat > dumpdir/%f"</userinput>
+</screen>
+  </para>
+
+  <para>
+   To use pipe to dump a database into a directory-format archive
+   in parallel with 5 worker jobs (the directory <literal>dumpdir</literal> needs to exist
+   before running the command).
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump -Fd mydb -j 5 --pipe="cat > dumpdir/%f"</userinput>
+</screen>
+  </para>
+
+  <para>
+   To use pipe to compress and dump a database into a
+   directory-format archive (the directory <literal>dumpdir</literal> needs to
+   exist before running the command).
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump -Fd mydb --pipe="gzip > dumpdir/%f.gz"</userinput>
+</screen>
+  </para>
+
   <para>
    To reload an archive file into a (freshly created) database named
    <literal>newdb</literal>:
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 5e77ddd556f..6db9cbc12af 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -118,7 +118,10 @@ PostgreSQL documentation
        <para>
        Specifies the location of the archive file (or directory, for a
        directory-format archive) to be restored.
-       If not specified, the standard input is used.
+       This option and <option>--pipe</option> can't be set
+       at the same time.
+       If neither this option nor <option>--pipe</option> is specified,
+       the standard input is used.
        </para>
       </listitem>
      </varlistentry>
@@ -919,6 +922,32 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--pipe</option></term>
+      <listitem>
+       <para>
+        This option is only supported with the directory output
+        format. It can be used to read from multiple streams which
+        otherwise would not be possible with the directory mode.
+        For each stream, it starts a process which runs the
+        specified command and pipes its output to the pg_restore process.
+        This option is not valid if <option>filename</option> is also specified.
+       </para>
+       <para>
+        The pipe can be used to perform operations like
+        decompress using a custom algorithm, filter, or read from
+        a cloud storage. When reading from the pg_dump output,
+        the user would need a way to read the correct file in each
+        stream. To handle that, the pipe command supports a format
+        specifier %f. And all the instances of %f in the command string
+        will be replaced with the corresponding file name which
+        would have been used in the directory mode with <option>filename</option>.
+        This is same as the <option>--pipe</option> of pg-dump.
+        See <xref linkend="app-pgrestore-examples"/> below.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
        <term><option>--section=<replaceable class="parameter">sectionname</replaceable></option></term>
        <listitem>
@@ -1364,6 +1393,43 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 <prompt>$</prompt> <userinput>pg_restore -L db.list db.dump</userinput>
 </screen></para>
 
+  <para>
+   To use pg_restore with pipe to recreate from a dump in
+   directory-archive format. The database should not exist beforehand.
+   Assume in this example that the dump in directory-archive format is
+   stored in <literal>dumpdir</literal>.
+
+<screen>
+<prompt>$</prompt> <userinput>pg_restore -C -Fd -d postgres --pipe="cat dumpdir/%f"</userinput>
+</screen>
+  </para>
+
+  <para>
+   To use pg_restore with pipe to first decompress and then
+   recreate from a dump in directory-archive format. The database
+   should not exist beforehand.
+   Assume in this example that the dump in directory-archive format is
+   stored in <literal>dumpdir</literal>. And all files are
+   <literal>gzip</literal> compressed.
+
+<screen>
+<prompt>$</prompt> <userinput>pg_restore -C -Fd -d postgres --pipe="cat dumpdir/%f.gz | gunzip"</userinput>
+</screen>
+  </para>
+
+  <para>
+   To use pipe along with <option>-L</option> to recreate only
+   selectd items from a dump in the directory-archive format.
+   The database should not exist beforehand.
+   Assume in this example that the dump in directory-archive format is
+   stored in dumpdir.
+   The <literal>db.list</literal> file is the same as one used in the previous example with <option>-L</option>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_restore -C -Fd -d postgres --pipe="cat dumpdir/%f" -L db.list</userinput>
+</screen>
+  </para>
+
  </refsect1>
 
  <refsect1>
-- 
2.54.0.563.g4f69b47b94-goog