Re: pg_waldump: support decoding of WAL inside tarfile

amul sul <sulamul@gmail.com>

From: Amul Sul <sulamul@gmail.com>

To: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>

Date: 2025-08-25T12:28:44Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Move tar detection and compression logic to common.
- c8a350a43982 19 (unreleased) landed
pg_verifybackup: Enable WAL parsing for tar-format backups
- b3cf461b3cf9 19 (unreleased) landed
pg_waldump: Add support for reading WAL from tar archives
- b15c1513984e 19 (unreleased) landed
pg_waldump: Preparatory refactoring for tar archive WAL decoding.
- f8a0cd267170 19 (unreleased) landed
pg_verifybackup: Verify tar-format backups.
- 8dfd31290279 18.0 cited

Attachments

v2-0001-Refactor-pg_waldump-Move-some-declarations-to-new.patch (application/x-patch) patch v2-0001
v2-0002-Refactor-pg_waldump-Separate-logic-used-to-calcul.patch (application/x-patch) patch v2-0002
v2-0003-Refactor-pg_waldump-Restructure-TAP-tests.patch (application/x-patch) patch v2-0003
v2-0004-pg_waldump-Rename-directory-creation-routine-for-.patch (application/x-patch) patch v2-0004
v2-0005-pg_waldump-Add-support-for-archived-WAL-decoding.patch (application/x-patch) patch v2-0005
v2-0006-WIP-pg_waldump-Remove-the-restriction-on-the-orde.patch (application/x-patch) patch v2-0006
v2-0007-pg_verifybackup-Delay-default-WAL-directory-prepa.patch (application/x-patch) patch v2-0007
v2-0008-pg_verifybackup-Rename-the-wal-directory-switch-t.patch (application/x-patch) patch v2-0008
v2-0009-pg_verifybackup-enabled-WAL-parsing-for-tar-forma.patch (application/x-patch) patch v2-0009

On Thu, Aug 7, 2025 at 7:47 PM Amul Sul <sulamul@gmail.com> wrote:
> [....]
> -----------------------------------
> Known Issues & Status:
> -----------------------------------
> - Timeline Switching: The current implementation in patch 006 does not
> correctly handle timeline switching. This is a known issue, especially
> when a timeline change occurs on a WAL file that has been written to a
> temporary location.
>

This is still pending and will be addressed in the next version.
Therefore, patch 0006 remains marked as WIP.

> - Testing: Local regression tests on CentOS and macOS M4 are passing.
> However, some tests on macOS Sonoma (specifically 008_untar.pl and
> 010_client_untar.pl) are failing in the GitHub workflow with a "WAL
> parsing failed for timeline 1" error. This issue is currently being
> investigated.
>

This has been fixed in the attached version; all GitHub workflow tests
are now fine.

Regards,
Amul