Horton

Re: Minimal logical decoding on standbys

bdrouvotAWS <bdrouvot@amazon.com>

View thread
From: "Drouvot, Bertrand" <bdrouvot@amazon.com>
To: Ronan Dunklau <ronan.dunklau@aiven.io>, Ibrar Ahmed <ibrar.ahmad@gmail.com>, Amit Khandekar <amitdkhan.pg@gmail.com>, <pgsql-hackers@lists.postgresql.org>
Cc: Andres Freund <andres@anarazel.de>, <fabriziomello@gmail.com>, Alvaro Herrera <alvherre@2ndquadrant.com>, tushar <tushar.ahuja@enterprisedb.com>, "[pgdg] Robert Haas" <robertmhaas@gmail.com>, Rahila Syed <rahila.syed@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2021-08-02T15:31:46Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Reduce the log level in 035_standby_logical_decoding.pl.

  2. 035_standby_logical_decoding: Add missing waits for replication

  3. For cascading replication, wake physical and logical walsenders separately

  4. Handle logical slot conflicts on standby

  5. Support invalidating replication slots due to horizon and wal_level

  6. Prevent use of invalidated logical slot in CreateDecodingContext()

  7. Replace replication slot's invalidated_at LSN with an enum

  8. Pass down table relation into more index relation functions

  9. Assert only valid flag bits are passed to visibilitymap_set()

  10. Remove unused _bt_delitems_delete() argument.

  11. Add xl_btree_delete optimization.

Hi Ronan,

Thanks for looking at it.

On 8/2/21 1:57 PM, Ronan Dunklau wrote:
> Le mardi 27 juillet 2021, 09:23:48 CEST Drouvot, Bertrand a écrit :
>> Thanks for the warning, rebase done and new v21 version attached.
>>
>> Bertrand
> Hello,
>
> I've taken a look at this patch, and it looks like you adressed every prior
> remark, including the race condition Andres was worried about.

I think there is still 2 points that need to be addressed (see [1])

>
> As for the basics: make check-world and make installcheck-world pass.
>
> I think the beahviour when dropping a database on the primary should be
> documented, and proper procedures for handling it correctly should be
> suggested.
>
> Something along the lines of:
>
> "If a database is dropped on the primary server, the logical replication slot
> on the standby will be dropped as well. This means that you should ensure that
> the client usually connected to this slot has had the opportunity to stream
> the latest changes before the database is dropped."

I am not sure we should highlight this as part of this patch.

I mean, the same would currently occur on a non standby if you drop a 
database that has a replication slot linked to it.

>
> As for the patches themselves, I only have two small comments to make.
>
> In patch 0002, in InvalidateConflictingLogicalReplicationSlots, I don't see the
> need to check for an InvalidOid since we already check the SlotIsLogical:
>
> +               /* We are only dealing with *logical* slot conflicts. */
> +               if (!SlotIsLogical(s))
> +                       continue;
> +
> +               /* not our database and we don't want all the database,
> skip */
> +               if ((s->data.database != InvalidOid && s->data.database
> != dboid) && TransactionIdIsValid(xid))
> +                       continue;

Agree, v22 attached in [1] does remove the useless s->data.database != 
InvalidOid check, thanks!

>
> In patch 0004, small typo in the test file:
> +##################################################
> +# Test standby promotion and logical decoding bheavior
> +# after the standby gets promoted.
> +##################################################

Typo also fixed in v22, thanks!

Bertrand

[1]: 
https://www.postgresql.org/message-id/69aad0bf-697a-04e1-df6c-0920ec8fa528%40amazon.com