Re: Perform streaming logical transactions by background workers and parallel apply
Peter Smith <smithpb2250@gmail.com>
Here are my review comments for patch v87-0002.
======
doc/src/sgml/config.sgml
1.
<para>
- Allows streaming or serializing changes immediately in
logical decoding.
The allowed values of <varname>logical_replication_mode</varname> are
- <literal>buffered</literal> and <literal>immediate</literal>. When set
- to <literal>immediate</literal>, stream each change if
+ <literal>buffered</literal> and <literal>immediate</literal>.
The default
+ is <literal>buffered</literal>.
+ </para>
I didn't think it was necessary to say “of logical_replication_mode”.
IMO that much is already obvious because this is the first sentence of
the description for logical_replication_mode.
(see also review comment #4)
~~~
2.
+ <para>
+ On the publisher side, it allows streaming or serializing changes
+ immediately in logical decoding. When set to
+ <literal>immediate</literal>, stream each change if
<literal>streaming</literal> option (see optional parameters set by
<link linkend="sql-createsubscription"><command>CREATE
SUBSCRIPTION</command></link>)
is enabled, otherwise, serialize each change. When set to
- <literal>buffered</literal>, which is the default, decoding will stream
- or serialize changes when <varname>logical_decoding_work_mem</varname>
- is reached.
+ <literal>buffered</literal>, decoding will stream or serialize changes
+ when <varname>logical_decoding_work_mem</varname> is reached.
</para>
2a.
"it allows" --> "logical_replication_mode allows"
2b.
"decoding" --> "the decoding"
~~~
3.
+ <para>
+ On the subscriber side, if <literal>streaming</literal> option is set
+ to <literal>parallel</literal>, this parameter also allows the leader
+ apply worker to send changes to the shared memory queue or to serialize
+ changes. When set to <literal>buffered</literal>, the leader sends
+ changes to parallel apply workers via shared memory queue. When set to
+ <literal>immediate</literal>, the leader serializes all changes to
+ files and notifies the parallel apply workers to read and apply them at
+ the end of the transaction.
+ </para>
"this parameter also allows" --> "logical_replication_mode also allows"
~~~
4.
<para>
This parameter is intended to be used to test logical decoding and
replication of large transactions for which otherwise we need to
generate the changes till <varname>logical_decoding_work_mem</varname>
- is reached.
+ is reached. Moreover, this can also be used to test the transmission of
+ changes between the leader and parallel apply workers.
</para>
"Moreover, this can also" --> "It can also"
I am wondering would this sentence be better put at the top of the GUC
description. So then the first paragraph becomes like this:
SUGGESTION (I've also added another sentence "The effect of...")
The allowed values are buffered and immediate. The default is
buffered. This parameter is intended to be used to test logical
decoding and replication of large transactions for which otherwise we
need to generate the changes till logical_decoding_work_mem is
reached. It can also be used to test the transmission of changes
between the leader and parallel apply workers. The effect of
logical_replication_mode is different for the publisher and
subscriber:
On the publisher side...
On the subscriber side...
======
.../replication/logical/applyparallelworker.c
5.
+ /*
+ * In immeidate mode, directly return false so that we can switch to
+ * PARTIAL_SERIALIZE mode and serialize remaining changes to files.
+ */
+ if (logical_replication_mode == LOGICAL_REP_MODE_IMMEDIATE)
+ return false;
Typo "immediate"
Also, I felt "directly" is not needed. "return false" and "directly
return false" is the same.
SUGGESTION
Using ‘immediate’ mode returns false to cause a switch to
PARTIAL_SERIALIZE mode so that the remaining changes will be
serialized.
======
src/backend/utils/misc/guc_tables.c
6.
{
{"logical_replication_mode", PGC_USERSET, DEVELOPER_OPTIONS,
- gettext_noop("Allows streaming or serializing each change in logical
decoding."),
- NULL,
+ gettext_noop("Controls the behavior of logical replication publisher
and subscriber"),
+ gettext_noop("If set to immediate, on the publisher side, it "
+ "allows streaming or serializing each change in "
+ "logical decoding. On the subscriber side, in "
+ "parallel streaming mode, it allows the leader apply "
+ "worker to serialize changes to files and notifies "
+ "the parallel apply workers to read and apply them at "
+ "the end of the transaction."),
GUC_NOT_IN_SAMPLE
},
6a. short description
User PoV behaviour should be the same. Instead, maybe say "controls
the internal behavior" or something like that?
~
6b. long description
IMO the long description shouldn’t mention ‘immediate’ mode first as it does.
BEFORE
If set to immediate, on the publisher side, ...
AFTER
On the publisher side, ...
------
Kind Regards,
Peter Smith.
Fujitsu Australia
Commits
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Fix invalid memory access during the shutdown of the parallel apply worker.
- 3d144c6c8602 16.0 landed
-
Fix assertion failure in apply worker.
- de63f8dadee4 16.0 landed
-
Use elog to report unexpected action in handle_streamed_transaction().
- 781ac42d43ab 16.0 landed
-
Use appropriate wait event when sending data in the apply worker.
- d9d7fe68d35e 16.0 landed
-
Allow the logical_replication_mode to be used on the subscriber.
- 9f2213a7c575 16.0 landed
-
Rename GUC logical_decoding_mode to logical_replication_mode.
- 1e8b61735cfb 16.0 landed
-
Display the leader apply worker's PID for parallel apply workers.
- d540a02a724b 16.0 landed
-
Improve the code to decide and process the apply action.
- c981d9145dea 16.0 landed
-
Document the newly added wait events added by commit 216a784829.
- cd06ccd78fcf 16.0 landed
-
Perform apply of large transactions by parallel workers.
- 216a784829c2 16.0 landed
-
Wake up a subscription's replication worker processes after DDL.
- c6e1f62e2cee 16.0 cited
-
Add copyright notices to meson files
- 8284cf5f746f 16.0 cited
-
Better document logical replication parameters
- a8500750ca0a 16.0 cited
-
Add a common function to generate the origin name.
- 776e1c8a5d14 16.0 cited
-
Harmonize parameter names in storage and AM code.
- bfcf1b34805f 16.0 cited
-
Avoid using list_length() to test for empty list.
- efd0c16becbf 16.0 cited
-
Improve two comments related to a boolean DefElem's value
- 8445f5a21d40 16.0 cited
-
Fix partition table's REPLICA IDENTITY checking on the subscriber.
- 26b3455afaa9 15.0 cited
-
Fix data inconsistency between publisher and subscriber.
- b7658c24c7c1 15.0 cited
-
Fix cache look-up failures while applying changes in logical replication.
- 5a97b1325453 15.0 cited