Horton

Re: Minimal logical decoding on standbys

Ronan Dunklau <ronan.dunklau@aiven.io>

View thread
From: Ronan Dunklau <ronan.dunklau@aiven.io>
To: Ibrar Ahmed <ibrar.ahmad@gmail.com>, Amit Khandekar <amitdkhan.pg@gmail.com>, pgsql-hackers@lists.postgresql.org
Cc: Andres Freund <andres@anarazel.de>, fabriziomello@gmail.com, Alvaro Herrera <alvherre@2ndquadrant.com>, tushar <tushar.ahuja@enterprisedb.com>, "[pgdg] Robert Haas" <robertmhaas@gmail.com>, Rahila Syed <rahila.syed@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, "Drouvot, Bertrand" <bdrouvot@amazon.com>
Date: 2021-08-02T11:57:22Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Reduce the log level in 035_standby_logical_decoding.pl.

  2. 035_standby_logical_decoding: Add missing waits for replication

  3. For cascading replication, wake physical and logical walsenders separately

  4. Handle logical slot conflicts on standby

  5. Support invalidating replication slots due to horizon and wal_level

  6. Prevent use of invalidated logical slot in CreateDecodingContext()

  7. Replace replication slot's invalidated_at LSN with an enum

  8. Pass down table relation into more index relation functions

  9. Assert only valid flag bits are passed to visibilitymap_set()

  10. Remove unused _bt_delitems_delete() argument.

  11. Add xl_btree_delete optimization.

Le mardi 27 juillet 2021, 09:23:48 CEST Drouvot, Bertrand a écrit :
> Thanks for the warning, rebase done and new v21 version attached.
> 
> Bertrand

Hello,

I've taken a look at this patch, and it looks like you adressed every prior 
remark, including the race condition Andres was worried about.

As for the basics: make check-world and make installcheck-world pass. 

I think the beahviour when dropping a database on the primary should be 
documented, and proper procedures for handling it correctly should be 
suggested.

Something along the lines of:

"If a database is dropped on the primary server, the logical replication slot 
on the standby will be dropped as well. This means that you should ensure that 
the client usually connected to this slot has had the opportunity to stream 
the latest changes before the database is dropped."

As for the patches themselves, I only have two small comments to make.

In patch 0002, in InvalidateConflictingLogicalReplicationSlots, I don't see the 
need to check for an InvalidOid since we already check the SlotIsLogical:

+		/* We are only dealing with *logical* slot conflicts. */
+		if (!SlotIsLogical(s))
+			continue;
+
+		/* not our database and we don't want all the database, 
skip */
+		if ((s->data.database != InvalidOid && s->data.database 
!= dboid) && TransactionIdIsValid(xid))
+			continue;

In patch 0004, small typo in the test file:
+##################################################
+# Test standby promotion and logical decoding bheavior
+# after the standby gets promoted.
+##################################################

Thank you for working on this !

Regards,

-- 
Ronan Dunklau