Horton

Re: Logical replication timeout problem

Amit Kapila <amit.kapila16@gmail.com>

View thread View on postgr.es
From: Amit Kapila <amit.kapila16@gmail.com>
To: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: "wangw.fnst@fujitsu.com" <wangw.fnst@fujitsu.com>, "kuroda.hayato@fujitsu.com" <kuroda.hayato@fujitsu.com>, Peter Smith <smithpb2250@gmail.com>, Fabrice Chapuis <fabrice636861@gmail.com>, Simon Riggs <simon.riggs@enterprisedb.com>, Petr Jelinek <petr.jelinek@enterprisedb.com>, "tanghy.fnst@fujitsu.com" <tanghy.fnst@fujitsu.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Ajin Cherian <itsajin@gmail.com>
Date: 2022-03-17T10:14:31Z
Lists: pgsql-hackers 
On Thu, Mar 17, 2022 at 12:27 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Wed, Mar 16, 2022 at 7:38 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> >
> > After more thought, can we check only wal_sender_timeout without
> > skip-count? That is, in WalSndUpdateProgress(), if we have received
> > any reply from the subscriber in last (wal_sender_timeout / 2), we
> > don't need to do anything in terms of keep-alive. If not, we do
> > ProcessRepliesIfAny() (and probably WalSndCheckTimeOut()?) then
> > WalSndKeepalivesIfNecessary(). That way, we can send keep-alive
> > messages every (wal_sender_timeout / 2). And since we don't call them
> > for every change, we would not need to worry about the overhead much.
> >
>
> But won't that lead to a call to GetCurrentTimestamp() for each change
> we skip? IIUC from previous replies that lead to a slight slowdown in
> previous tests of Wang-San.
>

If the above is true then I think we can use a lower skip_count say 10
along with a timeout mechanism to send keepalive message. This will
help us to alleviate the overhead Wang-San has shown.

BTW, I think there could be one other advantage of using
ProcessRepliesIfAny() (as you are suggesting) is that it can help to
release sync waiters if there are any. I feel that would be the case
for the skip_empty_transactions patch [1] which uses
WalSndUpdateProgress to send keepalive messages after skipping empty
transactions.

[1] - https://www.postgresql.org/message-id/CAFPTHDYvRSyT5ppYSPsH4Ozs0_W62-nffu0%3DmY1%2BsVipF%3DUN-g%40mail.gmail.com

-- 
With Regards,
Amit Kapila.

Commits

  1. Fix the logical replication timeout during large DDLs.

  2. Fix the logical replication timeout during large transactions.

  3. Rethink the delay-checkpoint-end mechanism in the back-branches.

  4. Revert "Logical decoding of sequences"

  5. Skip empty transactions for logical replication.

  6. Allow specifying column lists for logical replication

  7. Add decoding of sequences to built-in replication

  8. Fix ABI break introduced by commit 4daa140a2f.

  9. Lag tracking for logical replication