v22-0002-Update-documentation.patch
application/octet-stream
Filename: v22-0002-Update-documentation.patch
Type: application/octet-stream
Part: 1
Message:
RE: speed up a logical replica setup
Patch
Same data as JSON:
GET /api/v1/attachments/:id/patch
the parsed metadata as JSON — format, series position, per-file stats; never the diff bytes.
API reference →
Format: format-patch
Series: patch v22-0002
Subject: Update documentation
| File | + | − |
|---|---|---|
| doc/src/sgml/ref/pg_createsubscriber.sgml | 142 | 63 |
From 0411bbf0cfe4bec6e4905421717fa097d055ce89 Mon Sep 17 00:00:00 2001
From: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Date: Tue, 13 Feb 2024 10:59:47 +0000
Subject: [PATCH v22 02/11] Update documentation
---
doc/src/sgml/ref/pg_createsubscriber.sgml | 205 +++++++++++++++-------
1 file changed, 142 insertions(+), 63 deletions(-)
diff --git a/doc/src/sgml/ref/pg_createsubscriber.sgml b/doc/src/sgml/ref/pg_createsubscriber.sgml
index f5238771b7..7cdd047d67 100644
--- a/doc/src/sgml/ref/pg_createsubscriber.sgml
+++ b/doc/src/sgml/ref/pg_createsubscriber.sgml
@@ -48,19 +48,99 @@ PostgreSQL documentation
</cmdsynopsis>
</refsynopsisdiv>
- <refsect1>
+ <refsect1 id="r1-app-pg_createsubscriber-1">
<title>Description</title>
<para>
- <application>pg_createsubscriber</application> creates a new logical
- replica from a physical standby server.
+ The <application>pg_createsubscriber</application> creates a new <link
+ linkend="logical-replication-subscription">subscriber</link> from a physical
+ standby server.
</para>
<para>
- The <application>pg_createsubscriber</application> should be run at the target
- server. The source server (known as publisher server) should accept logical
- replication connections from the target server (known as subscriber server).
- The target server should accept local logical replication connection.
+ The <application>pg_createsubscriber</application> must be run at the target
+ server. The source server (known as publisher server) must accept both
+ normal and logical replication connections from the target server (known as
+ subscriber server). The target server must accept normal local connections.
</para>
+
+ <para>
+ There are some prerequisites for both the source and target instance. If
+ these are not met an error will be reported.
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ The given target data directory must have the same system identifier than the
+ source data directory.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The target instance must be used as a physical standby.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The given database user for the target instance must have privileges for
+ creating subscriptions and using functions for replication origin.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The target instance must have
+ <link linkend="guc-max-replication-slots"><varname>max_replication_slots</varname></link>
+ and <link linkend="guc-max-logical-replication-workers"><varname>max_logical_replication_workers</varname></link>
+ configured to a value greater than or equal to the number of target
+ databases.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The target instance must have
+ <link linkend="guc-max-worker-processes"><varname>max_worker_processes</varname></link>
+ configured to a value greater than the number of target databases.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The source instance must have
+ <link linkend="guc-wal-level"><varname>wal_level</varname></link> as
+ <literal>logical</literal>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The target instance must have
+ <link linkend="guc-max-replication-slots"><varname>max_replication_slots</varname></link>
+ configured to a value greater than or equal to the number of target
+ databases and replication slots.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The target instance must have
+ <link linkend="guc-max-wal-senders"><varname>max_wal_senders</varname></link>
+ configured to a value greater than or equal to the number of target
+ databases and walsenders.
+ </para>
+ </listitem>
+ </itemizedlist>
+
+ <note>
+ <para>
+ After the successful conversion, a physical replication slot configured as
+ <link linkend="guc-primary-slot-name"><varname>primary_slot_name</varname></link>
+ would be removed from a primary instance.
+ </para>
+
+ <para>
+ The <application>pg_createsubscriber</application> focuses on large-scale
+ systems that contain more data than 1GB. For smaller systems, initial data
+ synchronization of <link linkend="logical-replication">logical
+ replication</link> is recommended.
+ </para>
+ </note>
</refsect1>
<refsect1>
@@ -191,7 +271,7 @@ PostgreSQL documentation
</refsect1>
<refsect1>
- <title>Notes</title>
+ <title>How It Works</title>
<para>
The transformation proceeds in the following steps:
@@ -200,97 +280,89 @@ PostgreSQL documentation
<procedure>
<step>
<para>
- <application>pg_createsubscriber</application> checks if the given target data
- directory has the same system identifier than the source data directory.
- Since it uses the recovery process as one of the steps, it starts the
- target server as a replica from the source server. If the system
- identifier is not the same, <application>pg_createsubscriber</application> will
- terminate with an error.
+ Checks the target can be converted. In particular, things listed in
+ <link linkend="r1-app-pg_createsubscriber-1">above section</link> would be
+ checked. If these are not met <application>pg_createsubscriber</application>
+ will terminate with an error.
</para>
</step>
<step>
<para>
- <application>pg_createsubscriber</application> checks if the target data
- directory is used by a physical replica. Stop the physical replica if it is
- running. One of the next steps is to add some recovery parameters that
- requires a server start. This step avoids an error.
+ Creates a publication and a logical replication slot for each specified
+ database on the source instance. These publications and logical replication
+ slots have generated names:
+ <quote><literal>pg_createsubscriber_%u</literal></quote> (parameters:
+ Database <parameter>oid</parameter>) for publications,
+ <quote><literal>pg_createsubscriber_%u_%d</literal></quote> (parameters:
+ Database <parameter>oid</parameter>, Pid <parameter>int</parameter>) for
+ replication slots.
</para>
</step>
-
<step>
<para>
- <application>pg_createsubscriber</application> creates one replication slot for
- each specified database on the source server. The replication slot name
- contains a <literal>pg_createsubscriber</literal> prefix. These replication
- slots will be used by the subscriptions in a future step. A temporary
- replication slot is used to get a consistent start location. This
- consistent LSN will be used as a stopping point in the <xref
- linkend="guc-recovery-target-lsn"/> parameter and by the
- subscriptions as a replication starting point. It guarantees that no
- transaction will be lost.
+ Stops the target instance. This is needed to add some recovery parameters
+ during the conversion.
</para>
</step>
-
<step>
<para>
- <application>pg_createsubscriber</application> writes recovery parameters into
- the target data directory and start the target server. It specifies a LSN
- (consistent LSN that was obtained in the previous step) of write-ahead
- log location up to which recovery will proceed. It also specifies
- <literal>promote</literal> as the action that the server should take once
- the recovery target is reached. This step finishes once the server ends
- standby mode and is accepting read-write operations.
+ Creates a temporary replication slot to get a consistent start location.
+ The slot has generated names:
+ <quote><literal>pg_createsubscriber_%d_startpoint</literal></quote>
+ (parameters: Pid <parameter>int</parameter>). Got consistent LSN will be
+ used as a stopping point in the <xref linkend="guc-recovery-target-lsn"/>
+ parameter and by the subscriptions as a replication starting point. It
+ guarantees that no transaction will be lost.
+ </para>
+ </step>
+ <step>
+ <para>
+ Writes recovery parameters into the target data directory and starts the
+ target instance. It specifies a LSN (consistent LSN that was obtained in
+ the previous step) of write-ahead log location up to which recovery will
+ proceed. It also specifies <literal>promote</literal> as the action that
+ the server should take once the recovery target is reached. This step
+ finishes once the server ends standby mode and is accepting read-write
+ operations.
</para>
</step>
<step>
<para>
- Next, <application>pg_createsubscriber</application> creates one publication
- for each specified database on the source server. Each publication
- replicates changes for all tables in the database. The publication name
- contains a <literal>pg_createsubscriber</literal> prefix. These publication
- will be used by a corresponding subscription in a next step.
+ Creates a subscription for each specified database on the target instance.
+ These subscriptions have generated name:
+ <quote><literal>pg_createsubscriber_%u_%d</literal></quote> (parameters:
+ Database <parameter>oid</parameter>, Pid <parameter>int</parameter>).
+ These subscription have same subscription options:
+ <quote><literal>create_slot = false, copy_data = false, enabled = false</literal></quote>.
</para>
</step>
<step>
<para>
- <application>pg_createsubscriber</application> creates one subscription for
- each specified database on the target server. Each subscription name
- contains a <literal>pg_createsubscriber</literal> prefix. The replication slot
- name is identical to the subscription name. It does not copy existing data
- from the source server. It does not create a replication slot. Instead, it
- uses the replication slot that was created in a previous step. The
- subscription is created but it is not enabled yet. The reason is the
- replication progress must be set to the consistent LSN but replication
- origin name contains the subscription oid in its name. Hence, the
- subscription will be enabled in a separate step.
+ Sets replication progress to the consistent LSN that was obtained in a
+ previous step. This is the exact LSN to be used as a initial location for
+ each subscription.
</para>
</step>
<step>
<para>
- <application>pg_createsubscriber</application> sets the replication progress to
- the consistent LSN that was obtained in a previous step. When the target
- server started the recovery process, it caught up to the consistent LSN.
- This is the exact LSN to be used as a initial location for each
- subscription.
+ Enables the subscription for each specified database on the target server.
+ The subscription starts streaming from the consistent LSN.
</para>
</step>
<step>
<para>
- Finally, <application>pg_createsubscriber</application> enables the subscription
- for each specified database on the target server. The subscription starts
- streaming from the consistent LSN.
+ Stops the standby server.
</para>
</step>
<step>
<para>
- <application>pg_createsubscriber</application> stops the target server to change
- its system identifier.
+ Updates a system identifier on the target server.
</para>
</step>
</procedure>
@@ -300,8 +372,15 @@ PostgreSQL documentation
<title>Examples</title>
<para>
- To create a logical replica for databases <literal>hr</literal> and
- <literal>finance</literal> from a physical replica at <literal>foo</literal>:
+ Here is an example of using <application>pg_createsubscriber</application>.
+ Before running the command, please make sure target server is stopped.
+<screen>
+<prompt>$</prompt> <userinput>pg_ctl -D /usr/local/pgsql/data stop</userinput>
+</screen>
+
+ Then run <application>pg_createsubscriber</application>. Below tries to
+ create subscriptions for databases <literal>hr</literal> and
+ <literal>finance</literal> from a physical standby:
<screen>
<prompt>$</prompt> <userinput>pg_createsubscriber -D /usr/local/pgsql/data -P "host=foo" -S "host=localhost" -d hr -d finance</userinput>
</screen>
--
2.43.0