v23-0002-Update-documentation.patch

application/octet-stream

Filename: v23-0002-Update-documentation.patch
Type: application/octet-stream
Part: 1
Message: Re: speed up a logical replica setup

Patch

Same data as JSON: GET /api/v1/attachments/:id/patch the parsed metadata as JSON — format, series position, per-file stats; never the diff bytes. API reference →
Format: format-patch
Series: patch v23-0002
Subject: Update documentation
File+
doc/src/sgml/ref/pg_createsubscriber.sgml 142 63
From 8ea3d757b925ab2b3c6e06f0a72155e788430e4e Mon Sep 17 00:00:00 2001
From: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Date: Tue, 13 Feb 2024 10:59:47 +0000
Subject: [PATCH v23 02/13] Update documentation

---
 doc/src/sgml/ref/pg_createsubscriber.sgml | 205 +++++++++++++++-------
 1 file changed, 142 insertions(+), 63 deletions(-)

diff --git a/doc/src/sgml/ref/pg_createsubscriber.sgml b/doc/src/sgml/ref/pg_createsubscriber.sgml
index f5238771b7..7cdd047d67 100644
--- a/doc/src/sgml/ref/pg_createsubscriber.sgml
+++ b/doc/src/sgml/ref/pg_createsubscriber.sgml
@@ -48,19 +48,99 @@ PostgreSQL documentation
   </cmdsynopsis>
  </refsynopsisdiv>
 
- <refsect1>
+ <refsect1 id="r1-app-pg_createsubscriber-1">
   <title>Description</title>
   <para>
-    <application>pg_createsubscriber</application> creates a new logical
-    replica from a physical standby server.
+   The <application>pg_createsubscriber</application> creates a new <link
+   linkend="logical-replication-subscription">subscriber</link> from a physical
+   standby server.
   </para>
 
   <para>
-   The <application>pg_createsubscriber</application> should be run at the target
-   server. The source server (known as publisher server) should accept logical
-   replication connections from the target server (known as subscriber server).
-   The target server should accept local logical replication connection.
+   The <application>pg_createsubscriber</application> must be run at the target
+   server. The source server (known as publisher server) must accept both
+   normal and logical replication connections from the target server (known as
+   subscriber server). The target server must accept normal local connections.
   </para>
+
+  <para>
+   There are some prerequisites for both the source and target instance. If
+   these are not met an error will be reported.
+  </para>
+
+  <itemizedlist>
+   <listitem>
+    <para>
+     The given target data directory must have the same system identifier than the
+     source data directory.
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+     The target instance must be used as a physical standby.
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+     The given database user for the target instance must have privileges for
+     creating subscriptions and using functions for replication origin.
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+     The target instance must have
+     <link linkend="guc-max-replication-slots"><varname>max_replication_slots</varname></link>
+     and <link linkend="guc-max-logical-replication-workers"><varname>max_logical_replication_workers</varname></link>
+     configured to a value greater than or equal to the number of target
+     databases.
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+     The target instance must have
+     <link linkend="guc-max-worker-processes"><varname>max_worker_processes</varname></link>
+     configured to a value greater than the number of target databases.
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+     The source instance must have
+     <link linkend="guc-wal-level"><varname>wal_level</varname></link> as
+     <literal>logical</literal>.
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+     The target instance must have
+     <link linkend="guc-max-replication-slots"><varname>max_replication_slots</varname></link>
+     configured to a value greater than or equal to the number of target
+     databases and replication slots.
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+     The target instance must have
+     <link linkend="guc-max-wal-senders"><varname>max_wal_senders</varname></link>
+     configured to a value greater than or equal to the number of target
+     databases and walsenders.
+    </para>
+   </listitem>
+  </itemizedlist>
+
+  <note>
+   <para>
+    After the successful conversion, a physical replication slot configured as
+    <link linkend="guc-primary-slot-name"><varname>primary_slot_name</varname></link>
+    would be removed from a primary instance.
+   </para>
+
+   <para>
+    The <application>pg_createsubscriber</application> focuses on large-scale
+    systems that contain more data than 1GB.  For smaller systems, initial data
+    synchronization of <link linkend="logical-replication">logical
+    replication</link> is recommended.
+   </para>
+  </note>
  </refsect1>
 
  <refsect1>
@@ -191,7 +271,7 @@ PostgreSQL documentation
  </refsect1>
 
  <refsect1>
-  <title>Notes</title>
+  <title>How It Works</title>
 
   <para>
    The transformation proceeds in the following steps:
@@ -200,97 +280,89 @@ PostgreSQL documentation
   <procedure>
    <step>
     <para>
-     <application>pg_createsubscriber</application> checks if the given target data
-     directory has the same system identifier than the source data directory.
-     Since it uses the recovery process as one of the steps, it starts the
-     target server as a replica from the source server. If the system
-     identifier is not the same, <application>pg_createsubscriber</application> will
-     terminate with an error.
+     Checks the target can be converted.  In particular, things listed in
+     <link linkend="r1-app-pg_createsubscriber-1">above section</link> would be
+     checked.  If these are not met <application>pg_createsubscriber</application>
+     will terminate with an error.
     </para>
    </step>
 
    <step>
     <para>
-     <application>pg_createsubscriber</application> checks if the target data
-     directory is used by a physical replica. Stop the physical replica if it is
-     running. One of the next steps is to add some recovery parameters that
-     requires a server start. This step avoids an error.
+     Creates a publication and a logical replication slot for each specified
+     database on the source instance.  These publications and logical replication
+     slots have generated names:
+     <quote><literal>pg_createsubscriber_%u</literal></quote> (parameters:
+     Database <parameter>oid</parameter>) for publications,
+     <quote><literal>pg_createsubscriber_%u_%d</literal></quote> (parameters:
+     Database <parameter>oid</parameter>, Pid <parameter>int</parameter>) for
+     replication slots.
     </para>
    </step>
-
    <step>
     <para>
-     <application>pg_createsubscriber</application> creates one replication slot for
-     each specified database on the source server. The replication slot name
-     contains a <literal>pg_createsubscriber</literal> prefix. These replication
-     slots will be used by the subscriptions in a future step.  A temporary
-     replication slot is used to get a consistent start location. This
-     consistent LSN will be used as a stopping point in the <xref
-     linkend="guc-recovery-target-lsn"/> parameter and by the
-     subscriptions as a replication starting point. It guarantees that no
-     transaction will be lost.
+     Stops the target instance.  This is needed to add some recovery parameters
+     during the conversion.
     </para>
    </step>
-
    <step>
     <para>
-     <application>pg_createsubscriber</application> writes recovery parameters into
-     the target data directory and start the target server. It specifies a LSN
-     (consistent LSN that was obtained in the previous step) of write-ahead
-     log location up to which recovery will proceed. It also specifies
-     <literal>promote</literal> as the action that the server should take once
-     the recovery target is reached. This step finishes once the server ends
-     standby mode and is accepting read-write operations.
+     Creates a temporary replication slot to get a consistent start location.
+     The slot has generated names:
+     <quote><literal>pg_createsubscriber_%d_startpoint</literal></quote>
+     (parameters: Pid <parameter>int</parameter>).  Got consistent LSN will be
+     used as a stopping point in the <xref linkend="guc-recovery-target-lsn"/>
+     parameter and by the subscriptions as a replication starting point. It
+     guarantees that no transaction will be lost.
+    </para>
+   </step>
+   <step>
+    <para>
+     Writes recovery parameters into the target data directory and starts the
+     target instance.  It specifies a LSN (consistent LSN that was obtained in
+     the previous step) of write-ahead log location up to which recovery will
+     proceed. It also specifies <literal>promote</literal> as the action that
+     the server should take once the recovery target is reached. This step
+     finishes once the server ends standby mode and is accepting read-write
+     operations.
     </para>
    </step>
 
    <step>
     <para>
-     Next, <application>pg_createsubscriber</application> creates one publication
-     for each specified database on the source server. Each publication
-     replicates changes for all tables in the database. The publication name
-     contains a <literal>pg_createsubscriber</literal> prefix. These publication
-     will be used by a corresponding subscription in a next step.
+     Creates a subscription for each specified database on the target instance.
+     These subscriptions have generated name:
+     <quote><literal>pg_createsubscriber_%u_%d</literal></quote> (parameters:
+     Database <parameter>oid</parameter>, Pid <parameter>int</parameter>).
+     These subscription have same subscription options:
+     <quote><literal>create_slot = false, copy_data = false, enabled = false</literal></quote>.
     </para>
    </step>
 
    <step>
     <para>
-     <application>pg_createsubscriber</application> creates one subscription for
-     each specified database on the target server. Each subscription name
-     contains a <literal>pg_createsubscriber</literal> prefix. The replication slot
-     name is identical to the subscription name. It does not copy existing data
-     from the source server. It does not create a replication slot. Instead, it
-     uses the replication slot that was created in a previous step. The
-     subscription is created but it is not enabled yet. The reason is the
-     replication progress must be set to the consistent LSN but replication
-     origin name contains the subscription oid in its name. Hence, the
-     subscription will be enabled in a separate step.
+     Sets replication progress to the consistent LSN that was obtained in a
+     previous step.  This is the exact LSN to be used as a initial location for
+     each subscription.
     </para>
    </step>
 
    <step>
     <para>
-     <application>pg_createsubscriber</application> sets the replication progress to
-     the consistent LSN that was obtained in a previous step. When the target
-     server started the recovery process, it caught up to the consistent LSN.
-     This is the exact LSN to be used as a initial location for each
-     subscription.
+     Enables the subscription for each specified database on the target server.
+     The subscription starts streaming from the consistent LSN.
     </para>
    </step>
 
    <step>
     <para>
-     Finally, <application>pg_createsubscriber</application> enables the subscription
-     for each specified database on the target server. The subscription starts
-     streaming from the consistent LSN.
+     Stops the standby server.
     </para>
    </step>
 
    <step>
     <para>
-     <application>pg_createsubscriber</application> stops the target server to change
-     its system identifier.
+     Updates a system identifier on the target server.
     </para>
    </step>
   </procedure>
@@ -300,8 +372,15 @@ PostgreSQL documentation
   <title>Examples</title>
 
   <para>
-   To create a logical replica for databases <literal>hr</literal> and
-   <literal>finance</literal> from a physical replica at <literal>foo</literal>:
+   Here is an example of using <application>pg_createsubscriber</application>.
+   Before running the command, please make sure target server is stopped.
+<screen>
+<prompt>$</prompt> <userinput>pg_ctl -D /usr/local/pgsql/data stop</userinput>
+</screen>
+
+   Then run <application>pg_createsubscriber</application>. Below tries to
+   create subscriptions for databases <literal>hr</literal> and
+   <literal>finance</literal> from a physical standby:
 <screen>
 <prompt>$</prompt> <userinput>pg_createsubscriber -D /usr/local/pgsql/data -P "host=foo" -S "host=localhost" -d hr -d finance</userinput>
 </screen>
-- 
2.41.0.windows.3