Horton

Re: prevent immature WAL streaming

Amul Sul <sulamul@gmail.com>

View thread View on postgr.es
From: Amul Sul <sulamul@gmail.com>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: "Bossart, Nathan" <bossartn@amazon.com>, "Jakub.Wartak@tomtom.com" <Jakub.Wartak@tomtom.com>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, Ryo Matsumura <matsumura.ryo@fujitsu.com>, "andres@anarazel.de" <andres@anarazel.de>, "masao.fujii@oss.nttdata.com" <masao.fujii@oss.nttdata.com>, "mengjuan.cmj@alibaba-inc.com" <mengjuan.cmj@alibaba-inc.com>, "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>
Date: 2021-10-07T13:22:39Z
Lists: pgsql-hackers 
On Thu, 7 Oct 2021 at 6:41 PM, Alvaro Herrera <alvherre@alvh.no-ip.org>
wrote:

> On 2021-Oct-07, Amul Sul wrote:
>
> > While reading this commit (ff9f111bce24), wondered can't we skip
> > missingContrecPtr global variable declaration and calculate that from
> > abortedRecPtr value whenever it needed. IIUC, missingContrecPtr is the
> > next page to the page that abortedRecPtr contain and that can be
> > calculated as "abortedRecPtr + (XLOG_BLCKSZ - (abortedRecPtr %
> > XLOG_BLCKSZ))", thoughts? Please correct me if I'm missing something,
> > thanks.
>
> I don't think that works -- what if the missing record is not on the
> next page but on some future one?  Imagine an enormously large record
> that starts on segment 1, covers all of segment 2 and ends in segment 3.
> We could have flushed segment 2 already, so with your idea we would skip
> ahead only to that position, but really we need to skip all the way to
> the first page of segment 3.
>
> This is easier to imagine if you set wal segment size to 1 MB, but it is
> possible with the default segment size too, since commit records can be
> arbitrarily large, and "logical message" records as well.


Make sense, thanks for the explanation.

Regards,
Amul
-- 
Regards,
Amul Sul
EDB: http://www.enterprisedb.com

Commits

  1. Fix determination of broken LSN in OVERWRITTEN_CONTRECORD

  2. Fix instability in 026_overwrite_contrecord.pl test.

  3. Fix typo

  4. Change recently added test code for stability

  5. Remove unstable, unnecessary test; fix typo

  6. Repair two portability oversights of new test

  7. Fix WAL replay in presence of an incomplete record

  8. Document XLOG_INCLUDE_XID a little better

  9. Revert "Avoid creating archive status ".ready" files too early"

  10. Avoid creating archive status ".ready" files too early

  11. Fix scenario where streaming standby gets stuck at a continuation record.

  12. Teach xlogreader to follow timeline switches

  13. Forbid using pg_xlogfile_name() and pg_xlogfile_name_offset() during