Horton
Menu

Re: pg_waldump: support decoding of WAL inside tarfile

Andrew Dunstan <andrew@dunslane.net>

View thread
From: Andrew Dunstan <andrew@dunslane.net>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Michael Paquier <michael@paquier.xyz>, Amul Sul <sulamul@gmail.com>, Zsolt Parragi <zsolt.parragi@percona.com>, Robert Haas <robertmhaas@gmail.com>, Chao Li <li.evan.chao@gmail.com>, Jakub Wartak <jakub.wartak@enterprisedb.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2026-03-22T21:19:09Z
Lists: pgsql-hackers 
On Sun, Mar 22, 2026 at 2:17 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

> I wrote:
> > ... We can make this function far simpler
> > and more obviously correct if we just accept that we'll read a
> > WAL file completely before spilling it.  See my proposed
> > alternative to 0004, attached.
>
> Actually, we can make that better yet by not expecting
> get_archive_wal_entry to clean up after init_archive's
> failure to free all irrelevant hashtable entries.
> Better version attached.
>
>
>

Yeah, this looks good.  I know we also still need to do something about
rmtree trying to remove files we haven't closed. But what we have so far in
this set LGTM. If you want to push this I'm good, otherwise I'll look at it
tomorrow or Tuesday.

cheers

andrew

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Use size_t instead of Size in pg_waldump

  2. More tar portability adjustments.

  3. Further harden tests that might use not-so-compatible tar versions.

  4. Harden astreamer tar parsing logic against archives it can't handle.

  5. Fix pg_waldump/t/001_basic.pl with BSD tar on ZFS.

  6. Remove a low-value, high-risk optimization in pg_waldump.

  7. Fix misuse of simplehash.h hash operations in pg_waldump.

  8. Fix file descriptor leakages in pg_waldump.

  9. Fix poorly-sized buffers in astreamer compression modules.

  10. Remove read_archive_file()'s "count" parameter.

  11. Report detailed errors from XLogFindNextRecord() failures.

  12. Fix assorted bugs in archive_waldump.c.

  13. Remove nonfunctional tar file trailer size check.

  14. Fix finalization of decompressor astreamers.

  15. Move tar detection and compression logic to common.

  16. pg_verifybackup: Enable WAL parsing for tar-format backups

  17. pg_waldump: Add support for reading WAL from tar archives

  18. pg_waldump: Preparatory refactoring for tar archive WAL decoding.

  19. pg_waldump: Remove file-level global WalSegSz.

  20. pg_verifybackup: Verify tar-format backups.