Re: Minimal logical decoding on standbys

Amit Kapila <amit.kapila16@gmail.com>

From: Amit Kapila <amit.kapila16@gmail.com>

To: "Drouvot, Bertrand" <bertranddrouvot.pg@gmail.com>

Cc: Andres Freund <andres@anarazel.de>, Robert Haas <robertmhaas@gmail.com>, Jeff Davis <pgsql@j-davis.com>, Thomas Munro <thomas.munro@gmail.com>, Alvaro Herrera <alvherre@2ndquadrant.com>, Ibrar Ahmed <ibrar.ahmad@gmail.com>, Amit Khandekar <amitdkhan.pg@gmail.com>, fabriziomello@gmail.com, tushar <tushar.ahuja@enterprisedb.com>, Rahila Syed <rahila.syed@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, Melanie Plageman <melanieplageman@gmail.com>

Date: 2023-03-31T11:58:53Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Reduce the log level in 035_standby_logical_decoding.pl.
- 3034dc56ef4b 16.0 landed
035_standby_logical_decoding: Add missing waits for replication
- 57411c82ce86 16.0 landed
For cascading replication, wake physical and logical walsenders separately
- e101dfac3a53 16.0 landed
Handle logical slot conflicts on standby
- 26669757b6a7 16.0 landed
Support invalidating replication slots due to horizon and wal_level
- be87200efd93 16.0 landed
Prevent use of invalidated logical slot in CreateDecodingContext()
- 4397abd0a2af 16.0 landed
Replace replication slot's invalidated_at LSN with an enum
- 15f8203a5975 16.0 landed
Pass down table relation into more index relation functions
- 61b313e47eb9 16.0 landed
Assert only valid flag bits are passed to visibilitymap_set()
- a88a18b1250b 16.0 landed
Remove unused _bt_delitems_delete() argument.
- dc43492e46c7 14.0 cited
Add xl_btree_delete optimization.
- d2e5e20e5711 13.0 cited

On Fri, Mar 31, 2023 at 4:17 PM Drouvot, Bertrand
<bertranddrouvot.pg@gmail.com> wrote:
>

+ * This is needed for logical decoding on standby. Indeed the "problem" is that
+ * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up
+ * by walreceiver when new WAL has been flushed. Which means that typically
+ * walsenders will get woken up at the same time that the startup process
+ * will be - which means that by the time the logical walsender checks
+ * GetXLogReplayRecPtr() it's unlikely that the startup process
already replayed
+ * the record and updated XLogCtl->lastReplayedEndRecPtr.
+ *
+ * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case.

IIUC we are introducing condition variables as we can't rely on
current wait events because they will lead to spurious wakeups for
logical walsenders due to the below code in walreceiver:
XLogWalRcvFlush()
{
...
/* Signal the startup process and walsender that new WAL has arrived */
WakeupRecovery();
if (AllowCascadeReplication())
WalSndWakeup();

Is my understanding correct?

Can't we simply avoid waking up logical walsenders at this place and
rather wake them up at ApplyWalRecord() where the 0005 patch does
conditionvariable broadcast? Now, there doesn't seem to be anything
that distinguishes between logical and physical walsender but I guess
we can add a variable in WalSnd structure to identify it.

-- 
With Regards,
Amit Kapila.