Re: pg_recvlogical: Prevent flushed data from being re-sent after restarting replication
Fujii Masao <masao.fujii@gmail.com>
From: Fujii Masao <masao.fujii@gmail.com>
To: Mircea Cadariu <cadariu.mircea@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2025-11-19T14:54:02Z
Lists: pgsql-hackers
On Wed, Nov 19, 2025 at 11:36 PM Mircea Cadariu <cadariu.mircea@gmail.com> wrote: > > Hi, > > > Update: I added another test to the attached test-only patch. This new > test uses pg_terminate_backend to trigger reconnection. > > Assuming the tests are fully correct (your input appreciated on this) we > can use them to validate the solution. Thanks for testing! BTW, I reproduced the issue as follows: #1. Start the server #2. Start pg_recvlogical: $ pg_recvlogical -S myslot -d postgres --create-slot --start -f test.out #3. Insert data every second: $ psql =# create table t(i serial); =# insert into t values(default); =# \watch 1 #4. In a separate session, terminate walsender to force pg_recvlogical to restart replication: $ psql =# select pg_terminate_backend(pid) from pg_stat_replication; #5. Wait for pg_recvlogical to restart replication #6. You will see duplicate records written to the output file, for example: $ cat test.out BEGIN 798 table public.t: INSERT: i[integer]:42 COMMIT 798 BEGIN 799 table public.t: INSERT: i[integer]:43 COMMIT 799 BEGIN 792 table public.t: INSERT: i[integer]:36 COMMIT 792 BEGIN 793 table public.t: INSERT: i[integer]:37 COMMIT 793 BEGIN 794 table public.t: INSERT: i[integer]:38 COMMIT 794 BEGIN 795 table public.t: INSERT: i[integer]:39 COMMIT 795 BEGIN 796 table public.t: INSERT: i[integer]:40 COMMIT 796 BEGIN 797 table public.t: INSERT: i[integer]:41 COMMIT 797 BEGIN 798 table public.t: INSERT: i[integer]:42 COMMIT 798 BEGIN 799 table public.t: INSERT: i[integer]:43 COMMIT 799 With the patch applied, these duplicate records no longer appear in the pg_recvlogical output. Regards, -- Fujii Masao
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
pg_recvlogical: remove unnecessary OutputFsync() return value checks.
- b98cc4a14e53 19 (unreleased) landed
-
Add test for pg_recvlogical reconnection behavior.
- d89b1d817513 19 (unreleased) landed
-
Add a new helper function wait_for_file() to Utils.pm.
- 0b10969db610 19 (unreleased) landed
-
pg_recvlogical: Prevent flushed data from being re-sent.
- 41cbdab0ab6d 19 (unreleased) landed