Horton

Re: prevent immature WAL streaming

Alvaro Herrera <alvherre@alvh.no-ip.org>

View thread View on postgr.es
From: "alvherre@alvh.no-ip.org" <alvherre@alvh.no-ip.org>
To: Jakub Wartak <Jakub.Wartak@tomtom.com>
Cc: "Bossart, Nathan" <bossartn@amazon.com>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>, "mengjuan.cmj@alibaba-inc.com" <mengjuan.cmj@alibaba-inc.com>
Date: 2021-08-25T23:29:54Z
Lists: pgsql-hackers 
On 2021-Aug-25, Jakub Wartak wrote:

> In order to get reliable reproducer and get proper the fault injection
> instead of playing with really filling up fs, apparently one could
> substitute fd with fd of /dev/full using e.g. dup2() so that every
> write is going to throw this error too:

Oh, this is a neat trick that I didn't know about.  Thanks.

> After restarting master and inspecting standby - in all of those above
> 3 cases - the standby didn't inhibit the "invalid contrecord length"
> at least here, while without corruption this v02 patch it is
> notorious. So if it passes the worst-case code review assumptions I
> would be wondering if it shouldn't even be committed as it stands
> right now.

Well, Nathan is right that this patch isn't really closing the hole
because we aren't tracking segment boundaries that aren't "earliest" nor
"latest" at the moment of registration.  The simplistic approach seems
okay for the archive case, but not for the replication case.

I also noticed today (facepalm) that the patch doesn't work unless you
have archiving enabled, because the registration code is only invoked
when XLogArchivingActive().  Doh.  This part is easy to solve.  The
other doesn't look easy ...

-- 
Álvaro Herrera              Valdivia, Chile  —  https://www.EnterpriseDB.com/

Commits

  1. Fix determination of broken LSN in OVERWRITTEN_CONTRECORD

  2. Fix instability in 026_overwrite_contrecord.pl test.

  3. Fix typo

  4. Change recently added test code for stability

  5. Remove unstable, unnecessary test; fix typo

  6. Repair two portability oversights of new test

  7. Fix WAL replay in presence of an incomplete record

  8. Document XLOG_INCLUDE_XID a little better

  9. Revert "Avoid creating archive status ".ready" files too early"

  10. Avoid creating archive status ".ready" files too early

  11. Fix scenario where streaming standby gets stuck at a continuation record.

  12. Teach xlogreader to follow timeline switches

  13. Forbid using pg_xlogfile_name() and pg_xlogfile_name_offset() during