Horton

RE: Exit walsender before confirming remote flush in logical replication

Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>

View thread
From: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>
To: 'Kyotaro Horiguchi' <horikyota.ntt@gmail.com>
Cc: "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>, "amit.kapila16@gmail.com" <amit.kapila16@gmail.com>, "ashutosh.bapat.oss@gmail.com" <ashutosh.bapat.oss@gmail.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>
Date: 2022-12-26T12:27:24Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add wal_sender_shutdown_timeout GUC to limit shutdown wait for replication

  2. pg_upgrade: Add --copy option

  3. Prevent possibility of panics during shutdown checkpoint.

  4. Support clean switchover.

Dear Horiguchi-san,

> > Thus how about before entering an apply_delay, logrep worker sending a
> > kind of crafted feedback, which reports commit_data.end_lsn as
> > flushpos?  A little tweak is needed in send_feedback() but seems to
> > work..
> 
> Thanks for replying! I tested your saying but it could not work well...
> 
> I made PoC based on the latest time-delayed patches [1] for non-streaming case.
> Apply workers that are delaying applications send begin_data.final_lsn as recvpos
> and flushpos in send_feedback().

Maybe I misunderstood what you said... I have also found that sentPtr is not the actual sent
position, but the starting point of the next WAL. You can see the comment below.

```
/*
 * How far have we sent WAL already? This is also advertised in
 * MyWalSnd->sentPtr.  (Actually, this is the next WAL location to send.)
 */
static XLogRecPtr sentPtr = InvalidXLogRecPtr;
```

We must use end_lsn for crafting messages to cheat the walsender, but such records
are included in COMMIT, not in BEGIN for the non-streaming case.
But if workers are delayed in apply_handle_commit(), will they hold locks for database
objects for a long time and it causes another issue.

Best Regards,
Hayato Kuroda
FUJITSU LIMITED