Horton

Re: XLogReadRecord() error in XlogReadTwoPhaseData()

Thomas Munro <thomas.munro@gmail.com>

View thread View on postgr.es
From: Thomas Munro <thomas.munro@gmail.com>
To: Noah Misch <noah@leadboat.com>
Cc: pgbf@twiska.com, Tom Lane <tgl@sss.pgh.pa.us>, Michael Paquier <michael@paquier.xyz>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2022-01-16T09:19:30Z
Lists: pgsql-hackers 
On Sun, Jan 16, 2022 at 8:12 PM Noah Misch <noah@leadboat.com> wrote:
> For specifics of the kernel bug, see the attached test program.  In brief, the
> bug arises if one process is write()ing or pwrite()ing a file at about the
> same time that another process is read()ing or pread()ing the same.  POSIX
> says the reader should see the data as it existed before the write or the
> newly-written data.  On this kernel, the reader can see zeros instead.  That
> leads to the $SUBJECT failure.  PostgreSQL processes write out a given WAL
> block 20-30 times in ~10ms, and COMMIT PREPARED reads that block.  The writers
> aren't changing the bytes of interest to COMMIT PREPARED, but the zeros from
> the kernel bug yield the failure.  We could opt to work around that by writing
> only the not-already-written portion of a WAL block, but I doubt that's
> worthwhile unless it happens to be a performance win anyway.
>
> Separately, while I don't know of relevance to PostgreSQL, I was interested to
> see that CentOS 7 pwrite()/pread() fail to have the POSIX-required atomicity.

FWIW there was some related discussion over here:

https://www.postgresql.org/message-id/flat/17064-bb0d7904ef72add3%40postgresql.org

Commits

  1. Under has_wal_read_bug, skip recovery/t/032_relfilenode_reuse.pl.

  2. Under has_wal_read_bug, skip contrib/bloom/t/001_wal.pl.

  3. Use Test::Builder::todo_start(), replacing $::TODO.

  4. On sparc64+ext4, suppress test failures from known WAL read failure.

  5. Report any XLogReadRecord() error in XlogReadTwoPhaseData().