Re: Minimal logical decoding on standbys

bdrouvotAWS <bdrouvot@amazon.com>

From: "Drouvot, Bertrand" <bdrouvot@amazon.com>

To: Andres Freund <andres@anarazel.de>

Cc: Ibrar Ahmed <ibrar.ahmad@gmail.com>, Amit Khandekar <amitdkhan.pg@gmail.com>, <fabriziomello@gmail.com>, Alvaro Herrera <alvherre@2ndquadrant.com>, tushar <tushar.ahuja@enterprisedb.com>, "[pgdg] Robert Haas" <robertmhaas@gmail.com>, Rahila Syed <rahila.syed@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>

Date: 2021-08-06T11:27:46Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Reduce the log level in 035_standby_logical_decoding.pl.
- 3034dc56ef4b 16.0 landed
035_standby_logical_decoding: Add missing waits for replication
- 57411c82ce86 16.0 landed
For cascading replication, wake physical and logical walsenders separately
- e101dfac3a53 16.0 landed
Handle logical slot conflicts on standby
- 26669757b6a7 16.0 landed
Support invalidating replication slots due to horizon and wal_level
- be87200efd93 16.0 landed
Prevent use of invalidated logical slot in CreateDecodingContext()
- 4397abd0a2af 16.0 landed
Replace replication slot's invalidated_at LSN with an enum
- 15f8203a5975 16.0 landed
Pass down table relation into more index relation functions
- 61b313e47eb9 16.0 landed
Assert only valid flag bits are passed to visibilitymap_set()
- a88a18b1250b 16.0 landed
Remove unused _bt_delitems_delete() argument.
- dc43492e46c7 14.0 cited
Add xl_btree_delete optimization.
- d2e5e20e5711 13.0 cited

Attachments

v23-0005-Doc-changes-describing-details-about-logical-dec.patch (text/plain) patch v23-0005
v23-0004-New-TAP-test-for-logical-decoding-on-standby.patch (text/plain) patch v23-0004
v23-0003-Allow-logical-decoding-on-standby.patch (text/plain) patch v23-0003
v23-0002-Handle-logical-slot-conflicts-on-standby.patch (text/plain) patch v23-0002
v23-0001-Add-info-in-WAL-records-in-preparation-for-logic.patch (text/plain) patch v23-0001

Hi,

On 8/2/21 6:01 PM, Andres Freund wrote:
> While working on this I found a, somewhat substantial, issue:
>> If so, do you already have in mind a way to handle this? (I thought you
>> already had in mind a way to handle it so the question)
> Yes. I think we need to add a condition variable to be able to wait for
> WAL positions to change. Either multiple condition variables (one for
> the flush position, one for the replay position), or one that just
> changes more often. That way one can wait for apply without a race
> condition.
>
Thanks for the feedback.

Wouldn't a condition variable on the replay position be enough? I don't 
get why the proposed one on the flush position is needed.

>> if not, what kind of additional  tests would you like to see?
> A few catalog rows being removed (e.g. due to DELETE and then VACUUM
> *without* full) and a standby without hot_standby_feedback catching
> that.

Test added in v23 attached.

Thanks

Bertrand