Horton

Re: long-standing data loss bug in initial sync of logical replication

Benoit Lobréau <benoit.lobreau@dalibo.com>

View thread
From: Benoit Lobréau <benoit.lobreau@dalibo.com>
To: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>, Amit Kapila <amit.kapila16@gmail.com>
Cc: Shlok Kyal <shlok.kyal.oss@gmail.com>, vignesh C <vignesh21@gmail.com>, Nitin Motiani <nitinmotiani@google.com>, Andres Freund <andres@anarazel.de>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Masahiko Sawada <sawada.mshk@gmail.com>
Date: 2025-03-03T09:03:44Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Fix typo in test file name added in commit 4909b38af0.

  2. Fix data loss in logical replication.

  3. Avoid invalidating all RelationSyncCache entries on publication rename.

  4. Remove obsolete RECHECK keyword completely

  5. Backport BackgroundPsql perl test module

On 3/3/25 8:41 AM, Zhijie Hou (Fujitsu) wrote:
> A nitpick with the data for the Concurrent Transaction (2000) case. The results
> show that the HEAD's data appears worse than the patch data, which seems
> unusual. However, I confirmed that the details in the attachment are as expected,
> so, this seems to be a typo. (I assume you intended to use a
> decimal point instead of a comma in the data like (8,43500...))

Hi,

Argh, yes, sorry! I didn't pay enough attention and accidentally 
inverted the Patch and Head numbers in the last line when copying them 
from the ODS to the email to match the previous report layout.

The comma is due to how decimals are written in my language (comma 
instead of dot). I forgot to "translate" it.

Concurrent Txn |    Head (sec)    |    Patch (sec) | Degradation in %
---------------------------------------------------------------------
50             |   0.1797647      |   0.1920949    |  6.85907744957
100            |   0.3693029      |   0.3823425    |  3.53086856344
500            |   1.62265755     |   1.91427485   | 17.97158617972
1000           |   3.01388635     |   3.57678295   | 18.67676928162
2000           |   6.4713304      |   7.0171877    |  8.43500897435


> as Amit pointed out, we will share a new test script soon
> that uses the SQL API xxx_get_changes() to test. It would be great if you could
> verify the performance using the updated script as well.

Will do.

-- 
Benoit Lobréau
Consultant
http://dalibo.com