Re: Minimal logical decoding on standbys

Bertrand Drouvot <bertranddrouvot.pg@gmail.com>

From: "Drouvot, Bertrand" <bertranddrouvot.pg@gmail.com>

To: Robert Haas <robertmhaas@gmail.com>

Cc: Andres Freund <andres@anarazel.de>, Alvaro Herrera <alvherre@2ndquadrant.com>, Ibrar Ahmed <ibrar.ahmad@gmail.com>, Amit Khandekar <amitdkhan.pg@gmail.com>, fabriziomello@gmail.com, tushar <tushar.ahuja@enterprisedb.com>, Rahila Syed <rahila.syed@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>

Date: 2022-12-14T13:05:15Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Reduce the log level in 035_standby_logical_decoding.pl.
- 3034dc56ef4b 16.0 landed
035_standby_logical_decoding: Add missing waits for replication
- 57411c82ce86 16.0 landed
For cascading replication, wake physical and logical walsenders separately
- e101dfac3a53 16.0 landed
Handle logical slot conflicts on standby
- 26669757b6a7 16.0 landed
Support invalidating replication slots due to horizon and wal_level
- be87200efd93 16.0 landed
Prevent use of invalidated logical slot in CreateDecodingContext()
- 4397abd0a2af 16.0 landed
Replace replication slot's invalidated_at LSN with an enum
- 15f8203a5975 16.0 landed
Pass down table relation into more index relation functions
- 61b313e47eb9 16.0 landed
Assert only valid flag bits are passed to visibilitymap_set()
- a88a18b1250b 16.0 landed
Remove unused _bt_delitems_delete() argument.
- dc43492e46c7 14.0 cited
Add xl_btree_delete optimization.
- d2e5e20e5711 13.0 cited

Attachments

v31-0006-Fixing-Walsender-corner-case-with-logical-decodi.patch (text/plain) patch v31-0006
v31-0005-Doc-changes-describing-details-about-logical-dec.patch (text/plain) patch v31-0005
v31-0004-New-TAP-test-for-logical-decoding-on-standby.patch (text/plain) patch v31-0004
v31-0003-Allow-logical-decoding-on-standby.patch (text/plain) patch v31-0003
v31-0002-Handle-logical-slot-conflicts-on-standby.patch (text/plain) patch v31-0002
v31-0001-Add-info-in-WAL-records-in-preparation-for-logic.patch (text/plain) patch v31-0001

Hi,

On 12/13/22 5:37 PM, Drouvot, Bertrand wrote:
> Hi,
> 
> On 12/13/22 2:50 PM, Robert Haas wrote:
>> On Tue, Dec 13, 2022 at 5:49 AM Drouvot, Bertrand
> 
>> It seems kind of unfortunate to have to add payload to a whole bevy of
>> record types for this feature. I think it's worth it, both because the
>> feature is extremely important,
> 
> Agree and I don't think that there is other option than adding some payload in some WAL records (at the very beginning the proposal was to periodically log a new record
> that announces the current catalog xmin horizon).
> 
>> and also because there aren't any
>> record types that fall into this category that are going to be emitted
>> so frequently as to make it a performance problem. 
> 
> +1
> 
> If no objections from your side, I'll submit a patch proposal by tomorrow, which:
> 
> - get rid of IndexIsAccessibleInLogicalDecoding
> - let RelationIsAccessibleInLogicalDecoding deals with the index case
> - takes care of the padding where the new bool is added
> - convert this new bool to a flag for the xl_heap_visible case (adding a new bit to the already existing flag)
> - Add my proposed wording above to the commit message
> - Add your proposed wording above in a README

Please find attached v31 with the changes mentioned above (except that I put your wording into the commit message instead of a README: I think it helps to make
clear what the "design" for the patch series is).

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com