Re: Making pg_rewind faster

John H <johnhyvr@gmail.com>

From: John H <johnhyvr@gmail.com>

To: Japin Li <japinli@hotmail.com>

Cc: wenhui qiu <qiuwenhuifx@gmail.com>, Michael Paquier <michael@paquier.xyz>, Andres Freund <andres@anarazel.de>, Alexander Korotkov <aekorotkov@gmail.com>, Justin Kwan <justinpkwan@outlook.com>, Tom Lane <tgl@sss.pgh.pa.us>, pgsql-hackers <pgsql-hackers@postgresql.org>, vignesh <vignesh@cloudflare.com>, vignesh ravichandran <admin@viggy28.dev>, "hlinnaka@iki.fi" <hlinnaka@iki.fi>, "jkwan@cloudflare.com" <jkwan@cloudflare.com>

Date: 2025-07-03T19:59:29Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

pg_rewind: Skip copy of WAL segments generated before point of divergence
- 5173bfd0443e 19 (unreleased) landed
pg_rewind: Extend code detecting relation files to work with WAL files
- 6ae08d9583e9 19 (unreleased) landed
Split TESTDIR into TESTLOGDIR and TESTDATADIR
- c47885bd8b69 16.0 cited

Attachments

0006-Avoid-copying-WAL-segments-before-divergence-to-spee.patch (application/octet-stream) patch 0006

Hi,

On Wed, Jul 2, 2025 at 6:40 PM Japin Li <japinli@hotmail.com> wrote:
>
> >
>
> Splitting the logs from $PGDATA is definitely better. The question is whether
> it's worth implementing this directly in core or if a prominent note in the
> documentation would suffice.
>

I can work on the documentation update as a separate patch if folks
think this is worthwhile.

> >> On Wed, Jul 2, 2025 at 10:21 AM Japin Li <japinli@hotmail.com> wrote:
>
> Exactly!  It's confusing that getFileType() returns file_content_type_t
> instead of file_type_t.
>

Ah yes that is confusing, updated in patch.

> For v5 patch:
>
> 1.
> We could simply use the global WalSegSz variable within decide_file_action(),
> eliminating the need to pass wal_segsz_bytes as an argument.
>

Good point.

> 2.
> For last_common_segno, we could implement it similarly to WalSegSz, avoiding a
> signature change for decide_file_actions() and decide_file_action().  I'm not
> insisting on this approach, however.
>

I made it a global as well, and had to include access/xlog_internal.h
in pg_rewind.h but I don't feel strongly about it either way.

Thanks,

-- 
John Hsu - Amazon Web Services