Re: Minimal logical decoding on standbys
Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Commits
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Reduce the log level in 035_standby_logical_decoding.pl.
- 3034dc56ef4b 16.0 landed
-
035_standby_logical_decoding: Add missing waits for replication
- 57411c82ce86 16.0 landed
-
For cascading replication, wake physical and logical walsenders separately
- e101dfac3a53 16.0 landed
-
Handle logical slot conflicts on standby
- 26669757b6a7 16.0 landed
-
Support invalidating replication slots due to horizon and wal_level
- be87200efd93 16.0 landed
-
Prevent use of invalidated logical slot in CreateDecodingContext()
- 4397abd0a2af 16.0 landed
-
Replace replication slot's invalidated_at LSN with an enum
- 15f8203a5975 16.0 landed
-
Pass down table relation into more index relation functions
- 61b313e47eb9 16.0 landed
-
Assert only valid flag bits are passed to visibilitymap_set()
- a88a18b1250b 16.0 landed
-
Remove unused _bt_delitems_delete() argument.
- dc43492e46c7 14.0 cited
-
Add xl_btree_delete optimization.
- d2e5e20e5711 13.0 cited
Attachments
- v61-0006-Doc-changes-describing-details-about-logical-dec.patch (text/plain) patch v61-0006
- v61-0005-New-TAP-test-for-logical-decoding-on-standby.patch (text/plain) patch v61-0005
- v61-0004-For-cascading-replication-wake-up-physical-walse.patch (text/plain) patch v61-0004
- v61-0003-Allow-logical-decoding-on-standby.patch (text/plain) patch v61-0003
- v61-0002-Arrange-for-a-new-pg_stat_database_conflicts-and.patch (text/plain) patch v61-0002
- v61-0001-Handle-logical-slot-conflicts-on-standby.patch (text/plain) patch v61-0001
Hi,
On 4/5/23 1:59 PM, Amit Kapila wrote:
> On Wed, Apr 5, 2023 at 3:58 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>>
>> On Wed, Apr 5, 2023 at 2:41 PM Drouvot, Bertrand
>> <bertranddrouvot.pg@gmail.com> wrote:
>>
>> minor nitpick:
>> +
>> + /* Intentional fall through to session cancel */
>> + /* FALLTHROUGH */
>>
>> Do we need to repeat fall through twice in different ways?
>>
>
> Few minor comments on 0003:
> ========================
> 1.
> + case XLOG_PARAMETER_CHANGE:
> + {
> + xl_parameter_change *xlrec =
> + (xl_parameter_change *) XLogRecGetData(buf->record);
> +
> + /*
> + * If wal_level on primary is reduced to less than logical,
> + * then we want to prevent existing logical slots from being
> + * used. Existing logical slots on standby get invalidated
> + * when this WAL record is replayed; and further, slot
> + * creation fails when the wal level is not sufficient; but
> + * all these operations are not synchronized, so a logical
> + * slot may creep in while the wal_level is being reduced.
> + * Hence this extra check.
> + */
> + if (xlrec->wal_level < WAL_LEVEL_LOGICAL)
> + ereport(ERROR,
> + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
> + errmsg("logical decoding on standby requires wal_level to be at
> least logical on the primary server")));
>
> By looking at this change, it is not very clear that this can occur
> only on standby. I understand that on primary, we will not allow
> restarting the server after changing wal_level if there is a
> pre-existing slot but still this looks a bit odd. Shall we have an
> Assert to indicate that this will occur only on standby?
I think that's a fair point. Adding an Assert and a comment before the
Assert in V61 attached.
>
> 2.
> /*
> - * Since logical decoding is only permitted on a primary server, we know
> - * that the current timeline ID can't be changing any more. If we did this
> - * on a standby, we'd have to worry about the values we compute here
> - * becoming invalid due to a promotion or timeline change.
> + * Since logical decoding is also permitted on a standby server, we need
> + * to check if the server is in recovery to decide how to get the current
> + * timeline ID (so that it also cover the promotion or timeline change
> + * cases).
> */
> +
> + /* make sure we have enough WAL available */
> + flushptr = WalSndWaitForWal(targetPagePtr + reqLen);
> +
> + /* the standby could have been promoted, so check if still in recovery */
> + am_cascading_walsender = RecoveryInProgress();
>
> The first part of the comment explains why it is important to check
> RecoveryInProgress() and then immediately after that, the patch
> invokes WalSndWaitForWal(). It may be better to move the comment after
> WalSndWaitForWal() invocation.
Good catch, thanks! done in V61.
> Also, it will be better to write a
> comment as to why you need to do WalSndWaitForWal() before retrieving
> the current timeline as previously that was done afterward.
>
Agree, done in V61.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com