Horton

RE: long-standing data loss bug in initial sync of logical replication

Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>

View thread
From: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>
To: 'Shlok Kyal' <shlok.kyal.oss@gmail.com>
Cc: Masahiko Sawada <sawada.mshk@gmail.com>, vignesh C <vignesh21@gmail.com>, Nitin Motiani <nitinmotiani@google.com>, Andres Freund <andres@anarazel.de>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Amit Kapila <amit.kapila16@gmail.com>
Date: 2024-10-08T03:11:35Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Fix typo in test file name added in commit 4909b38af0.

  2. Fix data loss in logical replication.

  3. Avoid invalidating all RelationSyncCache entries on publication rename.

  4. Remove obsolete RECHECK keyword completely

  5. Backport BackgroundPsql perl test module

Attachments

Dear Shlok,

> I have tested this part. I observed that ,whenever we insert data in a
> partition table, the function 'get_rel_sync_entry' is called and a
> hash entry is created for the corresponding leaf node relid. So I feel
> while invalidating here we can specify 'PUBLICATION_PART_LEAF' . I
> have made the corresponding changes 0002 patch.

I also verified and it seems true. The root table is a virtual table and actual
changes are recorded in leaf ones. It is same for WAL layer. Logical decoding
obtains info from WAL records so leaf tables are passed to pgoutput layer as
"relation". I.e., I think it is enough to invalidate relcache of leaf.

> I have also modified the tests in 0001 patch. These changes are only
> related to syntax of writing tests.

LGTM. I found small improvements, please find the attached.

Best regards,
Hayato Kuroda
FUJITSU LIMITED