Horton

RE: Logical replication timeout problem

Wei Wang (Fujitsu) <wangw.fnst@fujitsu.com>

View thread View on postgr.es
From: "wangw.fnst@fujitsu.com" <wangw.fnst@fujitsu.com>
To: Ajin Cherian <itsajin@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Fabrice Chapuis <fabrice636861@gmail.com>, Simon Riggs <simon.riggs@enterprisedb.com>, Petr Jelinek <petr.jelinek@enterprisedb.com>, "tanghy.fnst@fujitsu.com" <tanghy.fnst@fujitsu.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2022-02-22T03:47:08Z
Lists: pgsql-hackers 
On Fri, Feb 18, 2022 at 10:51 AM Ajin Cherian <itsajin@gmail.com> wrote:
> Some comments:
Thanks for your review.

>  I see you only track skipped Inserts/Updates and Deletes. What about
> DDL operations that are skipped, what about truncate.
> What about changes made to unpublished tables? I wonder if you could
> create a test script that only did DDL operations
> and truncates, would this timeout happen?
According to your suggestion, I tested with DDL and truncate.
While testing, I ran only 20,000 DDLs and 10,000 truncations in one
transaction.
If I set wal_sender_timeout and wal_receiver_timeout to 30s, it will time out.
And if I use the default values, it will not time out.
IMHO there should not be long transactions that only contain DDL and
truncation. I'm not quite sure, do we need to handle this kind of use case?

Attach the test details.
[publisher-side]
configure:
    wal_sender_timeout = 30s or 60s
    wal_receiver_timeout = 30s or 60s
sql:
    create table tbl (a int primary key, b text);
    create table tbl2 (a int primary key, b text);
    create publication pub for table tbl;

[subscriber-side]
configure:
    wal_sender_timeout = 30s or 60s
    wal_receiver_timeout = 30s or 60s
sql:
    create table tbl (a int primary key, b text);"
    create subscription sub connection 'dbname=postgres user=postgres' publication pub;

[Execute sql in publisher-side]
In a transaction, execute the following SQL 10,000 times in a loop:
    alter table tbl2 rename column b to c;
    truncate table tbl2;
    alter table tbl2 rename column c to b;


Regards,
Wang wei

Commits

  1. Fix the logical replication timeout during large DDLs.

  2. Fix the logical replication timeout during large transactions.

  3. Rethink the delay-checkpoint-end mechanism in the back-branches.

  4. Revert "Logical decoding of sequences"

  5. Skip empty transactions for logical replication.

  6. Allow specifying column lists for logical replication

  7. Add decoding of sequences to built-in replication

  8. Fix ABI break introduced by commit 4daa140a2f.

  9. Lag tracking for logical replication