Horton

Re: Minimal logical decoding on standbys

Bertrand Drouvot <bertranddrouvot.pg@gmail.com>

View thread
From: "Drouvot, Bertrand" <bertranddrouvot.pg@gmail.com>
To: Andres Freund <andres@anarazel.de>, Robert Haas <robertmhaas@gmail.com>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>, Ibrar Ahmed <ibrar.ahmad@gmail.com>, Amit Khandekar <amitdkhan.pg@gmail.com>, fabriziomello@gmail.com, tushar <tushar.ahuja@enterprisedb.com>, Rahila Syed <rahila.syed@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2022-12-07T09:00:25Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Reduce the log level in 035_standby_logical_decoding.pl.

  2. 035_standby_logical_decoding: Add missing waits for replication

  3. For cascading replication, wake physical and logical walsenders separately

  4. Handle logical slot conflicts on standby

  5. Support invalidating replication slots due to horizon and wal_level

  6. Prevent use of invalidated logical slot in CreateDecodingContext()

  7. Replace replication slot's invalidated_at LSN with an enum

  8. Pass down table relation into more index relation functions

  9. Assert only valid flag bits are passed to visibilitymap_set()

  10. Remove unused _bt_delitems_delete() argument.

  11. Add xl_btree_delete optimization.

Attachments

Hi,

> Hi,
> Please find attached a new patch series:
> 
> v27-0001-Add-info-in-WAL-records-in-preparation-for-logic.patch
> v27-0002-Handle-logical-slot-conflicts-on-standby.patch
> v27-0003-Allow-logical-decoding-on-standby.patch
> v27-0004-New-TAP-test-for-logical-decoding-on-standby.patch
> v27-0005-Doc-changes-describing-details-about-logical-dec.patch
> v27-0006-Fixing-Walsender-corner-case-with-logical-decodi.patch
> 
> with the previous comments addressed, means mainly:
> 
> 1/ don't call table_open() in low-level functions in 0001: this is done with a new field "isusercatalog" in pg_index to indicate whether or not the index is linked to a table that has the storage parameter user_catalog_table set to true (we may want to make this field "invisible" though). This new field is then used in the new IndexIsAccessibleInLogicalDecoding Macro (through IndexIsUserCatalog).
> 
> 2/ Renaming the new field generated in the xlog record (to arrange conflict handling) from "onCatalogTable" to "onCatalogAccessibleInLogicalDecoding" to avoid any confusion (see 0001).
> 
> 3/ Making sure that "currTLI" is the current one in logical_read_xlog_page() (see 0003).
> 
> 4/ Fixing Walsender/startup process corner case: It's done in 0006 (I thought it is better to keep the other patches purely "feature" related and to address this corner case separately to ease the review). The fix is making use of a new
> 
> condition variable "replayedCV" so that the startup process can broadcast the walsender(s) once a replay is done.
> 
> Remarks:
> 
> - The new confl_active_logicalslot field added in pg_stat_database_conflicts (see 0002) is incremented only if the slot being invalidated is active (I think it makes more sense in regard to the other fields too). In all the cases (active/not active) the slot invalidation is reported in the logfile. The documentation update mentions this behavior (see 0002).
> 
> - LogStandbySnapshot() being moved outside of the loop in ReplicationSlotReserveWal() (see 0003), is a proposal made by Andres in [1] and I think it makes sense.
> 
> - Tap tests (see 0004) are covering: tests that the logical decoding on standby behaves correctly, conflicts, slots invalidations, standby promotion.
> 
> 
> Looking forward to your feedback,
> 
> 
> [1]: https://www.postgresql.org/message-id/20210406180231.qsnkyrgrm7gtxb73%40alap3.anarazel.de
> 

Please find attached v28 (mandatory rebase due to 8018ffbf58).

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com