Horton

Re: pg_waldump: support decoding of WAL inside tarfile

amul sul <sulamul@gmail.com>

View thread
From: Amul Sul <sulamul@gmail.com>
To: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2025-08-26T11:52:20Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Move tar detection and compression logic to common.

  2. pg_verifybackup: Enable WAL parsing for tar-format backups

  3. pg_waldump: Add support for reading WAL from tar archives

  4. pg_waldump: Preparatory refactoring for tar archive WAL decoding.

  5. pg_verifybackup: Verify tar-format backups.

Attachments

On Mon, Aug 25, 2025 at 5:58 PM Amul Sul <sulamul@gmail.com> wrote:
>
> On Thu, Aug 7, 2025 at 7:47 PM Amul Sul <sulamul@gmail.com> wrote:
> > [....]
> > -----------------------------------
> > Known Issues & Status:
> > -----------------------------------
> > - Timeline Switching: The current implementation in patch 006 does not
> > correctly handle timeline switching. This is a known issue, especially
> > when a timeline change occurs on a WAL file that has been written to a
> > temporary location.
> >
>
> This is still pending and will be addressed in the next version.
> Therefore, patch 0006 remains marked as WIP.
>

After testing pg_waldump, I have realised that my previous
understanding of its timeline handling was incorrect. I had mistakenly
assumed by reading xlogreader code that it would use the same
timeline-switching logic found in xlogreader, without first verifying
this behavior. In testing, I found that pg_waldump does not follow
timeline switches. Instead, it expects all WAL files to be from a
single timeline, which is either specified by the user or determined
from the starting segment or default 1.

This is a positive finding, as it means we don't need to make
significant changes to align pg_waldump's current behavior. The
attached patches are now complete and no longer works in progress --
read for review. Additionally, I've dropped patch v2-0004 because it is
no longer necessary. The primary patches that implement the proposed
feature are now 0004 and 0005 in the attached set.

Regards,
Amul