Horton

Re: Minimal logical decoding on standbys

Bertrand Drouvot <bertranddrouvot.pg@gmail.com>

View thread
From: "Drouvot, Bertrand" <bertranddrouvot.pg@gmail.com>
To: Andres Freund <andres@anarazel.de>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Jeff Davis <pgsql@j-davis.com>, Robert Haas <robertmhaas@gmail.com>, Thomas Munro <thomas.munro@gmail.com>, Ibrar Ahmed <ibrar.ahmad@gmail.com>, Amit Khandekar <amitdkhan.pg@gmail.com>, fabriziomello@gmail.com, tushar <tushar.ahuja@enterprisedb.com>, Rahila Syed <rahila.syed@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2023-04-07T18:27:51Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Reduce the log level in 035_standby_logical_decoding.pl.

  2. 035_standby_logical_decoding: Add missing waits for replication

  3. For cascading replication, wake physical and logical walsenders separately

  4. Handle logical slot conflicts on standby

  5. Support invalidating replication slots due to horizon and wal_level

  6. Prevent use of invalidated logical slot in CreateDecodingContext()

  7. Replace replication slot's invalidated_at LSN with an enum

  8. Pass down table relation into more index relation functions

  9. Assert only valid flag bits are passed to visibilitymap_set()

  10. Remove unused _bt_delitems_delete() argument.

  11. Add xl_btree_delete optimization.

Hi,

On 4/7/23 8:12 PM, Andres Freund wrote:
> Hi,
> 
> On 2023-04-07 08:47:57 -0700, Andres Freund wrote:
>> Integrated all of these.
> 
> Here's my current version. Changes:
> - Integrated Bertrand's changes
> - polished commit messages of 0001-0003
> - edited code comments for 0003, including
>    InvalidateObsoleteReplicationSlots()'s header
> - added a bump of SLOT_VERSION to 0001
> - moved addition of pg_log_standby_snapshot() to 0007
> - added a catversion bump for pg_log_standby_snapshot()
> - moved all the bits dealing with procsignals from 0003 to 0004, now the split
>    makes sense IMO
> - combined a few more sucessive ->safe_psql() calls
> 

Thanks!

> I see occasional failures in the tests, particularly in the new test using
> pg_authid, but not solely. cfbot also seems to have seen these:
> https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest%2F42%2F3740
> 
> I made a bogus attempt at a workaround for the pg_authid case last night. But
> that didn't actually fix anything, it just changed the timing.
> 
> I think the issue is that VACUUM does not force WAL to be flushed at the end
> (since it does not assign an xid). wait_for_replay_catchup() uses
> $node->lsn('flush'), which, due to VACUUM not flushing, can be an LSN from
> before VACUUM completed.
> 
> The problem can be made more likely by adding pg_usleep(1000000); before
> walwriter.c's call to XLogBackgroundFlush().
> 
> We probably should introduce some infrastructure in Cluster.pm for this, but
> for now I just added a 'flush_wal' table that we insert into after a
> VACUUM. That guarantees a WAL flush.
> 
> 
Ack for the Cluster.pm "improvement" and thanks for the "workaround"!

> I think some of the patches might have more reviewers than really applicable,
> and might also miss some. I'd appreciate if you could go over that...
> 

Sure, will do in a couple of hours.

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com