Re: Non-replayable WAL records through overflows and >MaxAllocSize lengths

On Sat, Apr 08, 2023 at 04:24:35PM +0200, Matthias van de Meent wrote:
> Thanks a lot! I'll post the separation of record construction and
> write-out to xlog in a future thread for 17.

Thanks!  Creating a new thread makes sense.

> One remaining question: Considering that the changes and checks of
> that commit are mostly internal to xloginsert.c (or xlog.c in older
> releases), and that no special public-facing changes were made, would
> it be safe to backport this to older releases?

The routine changes done in ffd1b6b cannot be backpatched on ABI
grounds, still you would propose to have protection around
needs_data as well as the whole record length.

> PostgreSQL 15 specifically would benefit from this as it supports
> external rmgrs which may generate WAL records and would benefit from
> these additional checks, but all supported releases of PostgreSQL have
> pg_logical_emit_message and are thus easily subject to the issue of
> writing oversized WAL records and subsequent recovery- and replication
> stream failures.

Custom RMGRs are a good argument, though I don't really see an urgent
argument about doing something in REL_15_STABLE.  For one, it would
mean more backpatching conflicts with ~14.  Another argument is that
XLogRecordMaxSize is not an exact science, either.  In ~15, a record
with a total size between XLogRecordMaxSize and
DecodeXLogRecordRequiredSpace(MaxAllocSize) would work, though it
would not in 16~ because we have the 4MB margin given as room for the
per-record allocation in the XLogReader.  A record of such a size
would not be generated anymore after a minor release update of 15.3~
if we were to do something about that by May on REL_15_STABLE.
--
Michael