Horton

Re: prevent immature WAL streaming

Alvaro Herrera <alvherre@alvh.no-ip.org>

View thread View on postgr.es
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
To: Hannu Krosing <hannuk@google.com>
Cc: "Bossart, Nathan" <bossartn@amazon.com>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, "andres@anarazel.de" <andres@anarazel.de>, "masao.fujii@oss.nttdata.com" <masao.fujii@oss.nttdata.com>, "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>, "mengjuan.cmj@alibaba-inc.com" <mengjuan.cmj@alibaba-inc.com>, "Jakub.Wartak@tomtom.com" <Jakub.Wartak@tomtom.com>, Ryo Matsumura <matsumura.ryo@fujitsu.com>
Date: 2021-09-29T14:48:11Z
Lists: pgsql-hackers 
On 2021-Sep-24, Hannu Krosing wrote:

Hi Hannu

> In the first email you write
> 
> > As mentioned in the course of thread [1], we're missing a fix for
> streaming replication to avoid sending records that the primary hasn't
> fully flushed yet.  This patch is a first attempt at fixing that problem
> by retreating the LSN reported as FlushPtr whenever a segment is
> registered, based on the understanding that if no registration exists
> then the LogwrtResult.Flush pointer can be taken at face value; but if a
> registration exists, then we have to stream only till the start LSN of
> that registered entry.
> 
> So did we end up holding back the wal_sender to not send anything that
> is not confirmed as flushed on master

No.  We eventually realized that that approach was a dead end, so I
abandoned the whole thing and attacked the problem differently.  So your
other questions don't apply.  I tried to make the commit message explain
both the problem and the solution in as much detail as possible; please
have a look at that and let me know if something is unclear.

Thanks

-- 
Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/
"El miedo atento y previsor es la madre de la seguridad" (E. Burke)

Commits

  1. Fix determination of broken LSN in OVERWRITTEN_CONTRECORD

  2. Fix instability in 026_overwrite_contrecord.pl test.

  3. Fix typo

  4. Change recently added test code for stability

  5. Remove unstable, unnecessary test; fix typo

  6. Repair two portability oversights of new test

  7. Fix WAL replay in presence of an incomplete record

  8. Document XLOG_INCLUDE_XID a little better

  9. Revert "Avoid creating archive status ".ready" files too early"

  10. Avoid creating archive status ".ready" files too early

  11. Fix scenario where streaming standby gets stuck at a continuation record.

  12. Teach xlogreader to follow timeline switches

  13. Forbid using pg_xlogfile_name() and pg_xlogfile_name_offset() during