Re: long-standing data loss bug in initial sync of logical replication

Tomas Vondra <tomas.vondra@enterprisedb.com>

From: Tomas Vondra <tomas.vondra@enterprisedb.com>

To: Amit Kapila <amit.kapila16@gmail.com>, Andres Freund <andres@anarazel.de>

Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>

Date: 2024-06-24T14:36:04Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Fix typo in test file name added in commit 4909b38af0.
- 50b8ad30f754 18.0 landed
- d96206f259d6 17.5 landed
- 9987c94662c2 16.9 landed
- 90bc4523fd47 15.13 landed
- bb1bc9fa962e 14.18 landed
- 4164d6976316 13.21 landed
Fix data loss in logical replication.
- 247ee94150b6 13.21 landed
- 4909b38af034 18.0 landed
- cadaf0ac4637 17.5 landed
- 9a2f8b4f01d5 16.9 landed
- 9f21be08e884 15.13 landed
- 0434033e8bb5 14.18 landed
Avoid invalidating all RelationSyncCache entries on publication rename.
- 3abe9dc18892 18.0 cited
Remove obsolete RECHECK keyword completely
- 7da1bdc2c2f1 18.0 cited
Backport BackgroundPsql perl test module
- 187b8991f70f 16.4 cited

On 6/24/24 12:54, Amit Kapila wrote:
> ...
>>
>>>> I'm not sure there are any cases where using SRE instead of AE would cause
>>>> problems for logical decoding, but it seems very hard to prove. I'd be very
>>>> surprised if just using SRE would not lead to corrupted cache contents in some
>>>> situations. The cases where a lower lock level is ok are ones where we just
>>>> don't care that the cache is coherent in that moment.
>>
>>> Are you saying it might break cases that are not corrupted now? How
>>> could obtaining a stronger lock have such effect?
>>
>> No, I mean that I don't know if using SRE instead of AE would have negative
>> consequences for logical decoding. I.e. whether, from a logical decoding POV,
>> it'd suffice to increase the lock level to just SRE instead of AE.
>>
>> Since I don't see how it'd be correct otherwise, it's kind of a moot question.
>>
> 
> We lost track of this thread and the bug is still open. IIUC, the
> conclusion is to use SRE in OpenTableList() to fix the reported issue.
> Andres, Tomas, please let me know if my understanding is wrong,
> otherwise, let's proceed and fix this issue.
> 

It's in the commitfest [https://commitfest.postgresql.org/48/4766/] so I
don't think we 'lost track' of it, but it's true we haven't done much
progress recently.

regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company