Horton
Menu

Re: Adding REPACK [concurrently]

Chao Li <li.evan.chao@gmail.com>

View thread
From: Chao Li <li.evan.chao@gmail.com>
To: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>
Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>, Amit Kapila <amit.kapila16@gmail.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, Antonin Houska <ah@cybertec.at>, Srinath Reddy Sadipiralla <srinath2133@gmail.com>, Mihail Nikalayeu <mihailnikalayeu@gmail.com>, Matthias van de Meent <boekewurm+postgres@gmail.com>, Pg Hackers <pgsql-hackers@lists.postgresql.org>, Robert Treat <rob@xzilla.net>
Date: 2026-04-21T07:24:25Z
Lists: pgsql-hackers 

> On Apr 10, 2026, at 18:53, Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com> wrote:
> 
> Hi,
> 
> When testing REPACK concurrently, I noticed that all WALs are retained from
> the moment REPACK begins copying data to the new table until the command
> finishes replaying concurrent changes on the new table and stops the repack
> decoding worker.
> 
> I understand the reason: the REPACK command itself starts a long-running
> transaction, and logical decoding does not advance restart_lsn beyond the
> oldest running transaction's start position. As a result, slot.restart_lsn
> remains unchanged, preventing the checkpointer from recycling WALs.
> 
> However, since REPACK can run for a long time (hours or even days), I'd like
> to confirm whether this is expected behavior or if we plan to improve it
> in the future ? And additionally, IIUC, REPACK without using concurrent option
> does not have this issue.
> 
> Given that we do not restart a REPACK, I think the repack decoding worker
> should be able to advance restart_lsn each time after writing changes
> (similar to how a physical slot behaves). To illustrate this, I've written
> a patch (attached) that implements this approach, and it works fine for me.
> 
> BTW, catalog_xmin also won't advance, but that seems not a big issue as
> the REPACK transaction itself also holds a snapshot that retains catalog tuples,
> so advancing catalog_xmin wouldn't change the situation anyway.
> 
> Thoughts ?
> 
> Best Regards,
> Hou zj
> <v1-0001-Allow-old-WALs-to-be-removed-during-REPACK-CONCUR.patch>

I found the same problem with LogicalConfirmReceivedLocation and posted a fix in a separate thread [1]. So I would withdraw my patch.

Looking at this patch, the change is exactly the same as what I did in [1], but I think the code comment should be updated as well. For the comment change, please see my patch in [1].

[1] https://www.postgresql.org/message-id/D8D9F770-DAA2-482C-A7E0-F87E5104C13E%40gmail.com

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Allow old WAL recycling during REPACK CONCURRENTLY

  2. Advance restart_lsn more eagerly in LogicalConfirmReceivedLocation

  3. Remove unnecessary signal handler change

  4. Improve REPACK (CONCURRENTLY) error messages some more

  5. Revert "Allow logical replication snapshots to be database-specific"

  6. Move REPACK (CONCURRENTLY) test out of stock regression tests

  7. REPACK: do not require REPLICATION or LOGIN

  8. Add missing initialization

  9. Simplify declaration of memcpy target

  10. Reserve replication slots specifically for REPACK

  11. doc: Add an example of REPACK (CONCURRENTLY)

  12. Allow logical replication snapshots to be database-specific

  13. Avoid different-size pointer-to-integer cast

  14. Fix valgrind failure

  15. Add CONCURRENTLY option to REPACK

  16. Rename cluster.c to repack.c (and corresponding .h)

  17. Allow index_create to suppress index_build progress reporting

  18. Make index_concurrently_create_copy more general

  19. Document the 'command' column of pg_stat_progress_repack

  20. Introduce the REPACK command

  21. Toggle logical decoding dynamically based on logical slot presence.

  22. Split vacuumdb to create vacuuming.c/h

  23. Remove ReorderBufferTupleBuf structure.

  24. Revert changes to CONCURRENTLY that "sped up" Xmin advance

  25. VACUUM: ignore indexing operations with CONCURRENTLY