Horton

Re: Making pg_rewind faster

Robert Haas <robertmhaas@gmail.com>

View thread
From: Robert Haas <robertmhaas@gmail.com>
To: Srinath Reddy Sadipiralla <srinath2133@gmail.com>
Cc: John H <johnhyvr@gmail.com>, wenhui qiu <qiuwenhuifx@gmail.com>, Japin Li <japinli@hotmail.com>, Michael Paquier <michael@paquier.xyz>, Andres Freund <andres@anarazel.de>, Alexander Korotkov <aekorotkov@gmail.com>, Justin Kwan <justinpkwan@outlook.com>, Tom Lane <tgl@sss.pgh.pa.us>, pgsql-hackers <pgsql-hackers@postgresql.org>, vignesh <vignesh@cloudflare.com>, vignesh ravichandran <admin@viggy28.dev>, "hlinnaka@iki.fi" <hlinnaka@iki.fi>, "jkwan@cloudflare.com" <jkwan@cloudflare.com>
Date: 2025-10-16T18:59:48Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. pg_rewind: Skip copy of WAL segments generated before point of divergence

  2. pg_rewind: Extend code detecting relation files to work with WAL files

  3. Split TESTDIR into TESTLOGDIR and TESTDATADIR

On Wed, Oct 15, 2025 at 10:27 AM Srinath Reddy Sadipiralla
<srinath2133@gmail.com> wrote:
>> On Fri, Oct 10, 2025 at 12:45 AM Srinath Reddy Sadipiralla
>> <srinath2133@gmail.com> wrote:
>> > XLogFilePath , then validate this new path with the given path
>> > ,this helps to catch invalid xlog files like pg_wal/00000001FFFFFFFFFFFFFF10.
>> > ...
>>
>> Are you concerned that somehow these files, which are named like XLog
>> files but actually
>> aren't, are somehow created therefore we should sync them in case?
>> I'm trying to understand how these files would be generated in the first place.
>
> the problem is not when server generates them because
> filename is properly calculated using the XLogRecPtr in
> XLogWrite->XLogFileInit->XLogFileInitInternal->XLogFilePath
> ,the main problem is when if someone manually places an invalid WAL file
> in pg_wal like 00000001FFFFFFFFFFFFFF10, IsXLogFileName will
> consider it as valid ,so with the approach as i mentioned earlier we can
> catch such cases.

I think that parsing the file name may be a good idea so that we can
do appropriate sanity checks on the values (e.g. checking that we're
only skipping copying prior to last_common_segno), but I do not think
we should worry too much about the user manually injecting invalid WAL
files. I mean, I would prefer that if that does happen, it either
works anyway or fails with a sensible error message, rather than
emitting an incomprehensible error message or dumping core. But, it is
in general true that if manual modifications are made to the data
directory, things may go terribly wrong, and this code is not obliged
to provide any more protection against such scenarios than we do in
other cases. Ultimately, such modifications are user error.

-- 
Robert Haas
EDB: http://www.enterprisedb.com