Re: Non-replayable WAL records through overflows and >MaxAllocSize lengths

Matthias van de Meent <boekewurm+postgres@gmail.com>

From: Matthias van de Meent <boekewurm+postgres@gmail.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Matthias van de Meent <boekewurm+postgres@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Andres Freund <andres@anarazel.de>, Ian Lawrence Barwick <barwick@gmail.com>, Heikki Linnakangas <hlinnaka@iki.fi>, David Zhang <david.zhang@highgo.ca>, Robert Haas <robertmhaas@gmail.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Date: 2023-04-06T23:50:00Z
Lists: pgsql-hackers
On Fri, 7 Apr 2023, 01:35 Michael Paquier, <michael@paquier.xyz> wrote:

> On Fri, Apr 07, 2023 at 08:08:34AM +0900, Michael Paquier wrote:
> > So bumping mainrdata_len to uint64 is actually not entirely in line
> > with this code.  Well, it will work because we'd still fail a couple
> > of lines down, but perhaps its readability should be improved so as
> > we have an extra check in this code path to make sure that
> > mainrdata_len is not higher than PG_UINT32_MAX, then use an
> > intermediate casted variable before saving the length in the record
> > data to make clear that the type of the main static length in
> > xloginsert.c is not the same as what a record has?  The v10 I sent
> > previously blocked this possibility, but not v11.
>

Yes, that was a bad oversight, which would've shown up in tests on a system
with an endianness that my computer doesn't have...


> So, I was thinking about something like the attached tweaking this
> point, the error details a bit, applying an indentation and writing a
> commit message...  Matthias?
>

That looks fine to me. Thanks for picking this up and fixing the issue.



Kind regards,

Matthias van de Meent

Commits

  1. Add more protections in WAL record APIs against overflows

  2. Add overflow protection for block-related data in WAL records

  3. Change internal RelFileNode references to RelFileNumber or RelFileLocator.

  4. Revamp the WAL record format.