Horton

Re: Minimal logical decoding on standbys

Rahila Syed <rahila.syed@2ndquadrant.com>

View thread
From: Rahila Syed <rahila.syed@2ndquadrant.com>
To: Amit Khandekar <amitdkhan.pg@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2020-01-10T12:20:10Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Reduce the log level in 035_standby_logical_decoding.pl.

  2. 035_standby_logical_decoding: Add missing waits for replication

  3. For cascading replication, wake physical and logical walsenders separately

  4. Handle logical slot conflicts on standby

  5. Support invalidating replication slots due to horizon and wal_level

  6. Prevent use of invalidated logical slot in CreateDecodingContext()

  7. Replace replication slot's invalidated_at LSN with an enum

  8. Pass down table relation into more index relation functions

  9. Assert only valid flag bits are passed to visibilitymap_set()

  10. Remove unused _bt_delitems_delete() argument.

  11. Add xl_btree_delete optimization.

Hi Amit,

Can you please rebase the patches as they don't apply on latest master?

Thank you,
Rahila Syed


On Thu, 26 Dec 2019 at 16:36, Amit Khandekar <amitdkhan.pg@gmail.com> wrote:

> On Tue, 24 Dec 2019 at 14:02, Amit Khandekar <amitdkhan.pg@gmail.com>
> wrote:
> >
> > On Thu, 19 Dec 2019 at 01:02, Rahila Syed <rahila.syed@2ndquadrant.com>
> wrote:
> > >
> > > Hi,
> > >
> > >> Hi, do you consistently get this failure on your machine ? I am not
> > >> able to get this failure, but I am going to analyze when/how this can
> > >> fail. Thanks
> > >>
> > > Yes, I am getting it each time I run make -C src/test/recovery/ check
> PROVE_TESTS=t/018_standby_logical_decoding_xmins.pl
> > > Also, there aren't any errors in logs indicating the cause.
> >
> > Thanks for the reproduction. Finally I could reproduce the behaviour.
> > It occurs once in 7-8 runs of the test on my machine. The issue is :
> > on master, the catalog_xmin does not immediately get updated. It
> > happens only after the hot standby feedback reaches on master. And I
> > haven't used wait_for_xmins() for these failing cases. I should use
> > that. Working on the same ...
>
> As mentioned above, I have used wait_for_xmins() so that we can wait
> for the xmins to be updated after hot standby feedback is processed.
> In one of the 3 scenarios where it failed for you, I removed the check
> at the second place because it was redundant. At the 3rd place, I did
> some appropriate changes with detailed comments. Please check.
> Basically we are checking that the master's phys catalog_xmin has
> advanced but not beyond standby's logical catalog_xmin. And for making
> sure the master's xmins are updated, I call txid_current() and then
> wait for the master's xmin to advance after hot-standby_feedback, and
> in this way I make sure the xmin/catalog_xmins are now up-to-date
> because of hot-standby-feedback, so that we can check whether the
> master's physical slot catalog_xmin has reached the value of standby's
> catalog_xmin but not gone past it.
>
> I have also moved the "wal_receiver_status_interval = 1" setting from
> master to standby. It was wrongly kept in master. This now reduces the
> test time by half, on my machine.
>
> Attached patch set v5 has only the test changes. Please check if now
> the test fails for you.
>
> >
> > --
> > Thanks,
> > -Amit Khandekar
> > EnterpriseDB Corporation
> > The Postgres Database Company
>
>
>
> --
> Thanks,
> -Amit Khandekar
> EnterpriseDB Corporation
> The Postgres Database Company
>


-- 
Rahila Syed
Performance Engineer
2ndQuadrant
http://www.2ndQuadrant.com <http://www.2ndquadrant.com/>
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services