Horton
Menu

Re: pg_waldump: support decoding of WAL inside tarfile

Robert Haas <robertmhaas@gmail.com>

View thread
From: Robert Haas <robertmhaas@gmail.com>
To: Amul Sul <sulamul@gmail.com>
Cc: Euler Taveira <euler@eulerto.com>, Chao Li <li.evan.chao@gmail.com>, Jakub Wartak <jakub.wartak@enterprisedb.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2026-02-07T13:04:32Z
Lists: pgsql-hackers 
On Wed, Feb 4, 2026 at 8:53 AM Amul Sul <sulamul@gmail.com> wrote:
> > 0008. I'm concerned about this patch. It is breaking backward compatibility if
> > you are using a long option (--wal-directory). Your proposal is a generic word
> > that represents both cases (file and directory). I agree. However, I wouldn't
> > remove --wal-directory from the tool. Instead, I would keep it with the same
> > short option ('w') but add a sentence saying this long option is deprecated and
> > will be removed in the future or even remove any traces of this long option
> > from the help and documentation but silently accept the old long option. I
> > prefer the latter because it is not a required argument so a deprecation
> > warning is not necessary IMO.
>
> Yeah, that was discussed with Robert offline and we believe that it is
> better to make it more generalized; since we can now use the same
> option to accept both wal-directory and wal-archived. pg_waldump has
> much more generic options for the same, such as -- path=PATH.

Of course, the fact that we discussed it doesn't mean that the issue
is completely settled. However, I don't think there would be general
support from other people on the project for the idea of getting rid
of the long option entirely, or even just the documentation for it. We
have long options for almost all short options these days, and I agree
with that as a general practice. Sometimes we have ONLY a long option,
but we very rarely have ONLY a short option, which is good because
sometimes we have tools with too many options for it to be viable to
give everything a short option, and the long options really help to
make things more self-documenting.

What I would consider a more viable option is to not do the rename and
leave this as --wal-directory even though the argument could really be
a directory or a file. That would avoid the backward compatibility
break that is troubling Euler. And, you could argue that a tar file is
enough like a directory that it won't really cause much confusion.
Personally, I favor renaming it. I think the number of people using
pg_waldump is fairly small, and the number of those people who are
using the long form of the option is very small. Hence, I don't
believe think the rename will inconvenience many users, and I think it
will improve clarity for future users. But, Euler (or someone else)
might take the opposite viewpoint.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Use size_t instead of Size in pg_waldump

  2. More tar portability adjustments.

  3. Further harden tests that might use not-so-compatible tar versions.

  4. Harden astreamer tar parsing logic against archives it can't handle.

  5. Fix pg_waldump/t/001_basic.pl with BSD tar on ZFS.

  6. Remove a low-value, high-risk optimization in pg_waldump.

  7. Fix misuse of simplehash.h hash operations in pg_waldump.

  8. Fix file descriptor leakages in pg_waldump.

  9. Fix poorly-sized buffers in astreamer compression modules.

  10. Remove read_archive_file()'s "count" parameter.

  11. Report detailed errors from XLogFindNextRecord() failures.

  12. Fix assorted bugs in archive_waldump.c.

  13. Remove nonfunctional tar file trailer size check.

  14. Fix finalization of decompressor astreamers.

  15. Move tar detection and compression logic to common.

  16. pg_verifybackup: Enable WAL parsing for tar-format backups

  17. pg_waldump: Add support for reading WAL from tar archives

  18. pg_waldump: Preparatory refactoring for tar archive WAL decoding.

  19. pg_waldump: Remove file-level global WalSegSz.

  20. pg_verifybackup: Verify tar-format backups.