Horton
Menu

Re: Adding REPACK [concurrently]

Antonin Houska <ah@cybertec.at>

View thread
From: Antonin Houska <ah@cybertec.at>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: Matthias van de Meent <boekewurm+postgres@gmail.com>, Mihail Nikalayeu <mihailnikalayeu@gmail.com>, Pg Hackers <pgsql-hackers@lists.postgresql.org>, Robert Treat <rob@xzilla.net>
Date: 2026-03-17T19:57:06Z
Lists: pgsql-hackers

Attachments

Antonin Houska <ah@cybertec.at> wrote:

> Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> 
> > On 2026-Mar-16, Matthias van de Meent wrote:
> > 
> > > On Mon, 16 Mar 2026 at 21:15, Antonin Houska <ah@cybertec.at> wrote:
> > 
> > > > Anyway (fortunately?), the concurrent use of slots by REPACK is limited
> > > > because, during the initialization of logical decoding, the backend needs to
> > > > wait for all the transactions having XID assigned to finish, and these include
> > > > the already running REPACK commands. See SnapBuildWaitSnapshot() and callers
> > > > if you're interested in details.
> > > 
> > > Huh, so would you be able to run more than one Repack Concurrently in
> > > the same database? ISTM that would not be possible, apart from
> > > possibly a mechanism comparable to the SAFE_IN_IC flag (to not wait on
> > > those backends).
> > 
> > Yeah, this sounds kind of bad news ...
> 
> Admittedly, it is a problem. I tried to address this in pg_squeeze by
> pre-allocating slots when it's clear (due to scheduling) that more than one
> table needs to be processed. This was an effort to achieve the best possible
> performance rather than a response to complaints of users about low
> throughput. Nevertheless, I'm glad I happened to mention it before it's too
> late.
> 
> Regarding solution, a flag like SAFE_IN_IC alone does not help. The
> information that particular transaction is used by REPACK (and therefore it
> does not have to be decoded) would need to be propagated to the
> xl_running_xacts WAL record too.

0007 in the next version tries to implement that.

-- 
Antonin Houska
Web: https://www.cybertec-postgresql.com

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Allow old WAL recycling during REPACK CONCURRENTLY

  2. Advance restart_lsn more eagerly in LogicalConfirmReceivedLocation

  3. Remove unnecessary signal handler change

  4. Improve REPACK (CONCURRENTLY) error messages some more

  5. Revert "Allow logical replication snapshots to be database-specific"

  6. Move REPACK (CONCURRENTLY) test out of stock regression tests

  7. REPACK: do not require REPLICATION or LOGIN

  8. Add missing initialization

  9. Simplify declaration of memcpy target

  10. Reserve replication slots specifically for REPACK

  11. doc: Add an example of REPACK (CONCURRENTLY)

  12. Allow logical replication snapshots to be database-specific

  13. Avoid different-size pointer-to-integer cast

  14. Fix valgrind failure

  15. Add CONCURRENTLY option to REPACK

  16. Rename cluster.c to repack.c (and corresponding .h)

  17. Allow index_create to suppress index_build progress reporting

  18. Make index_concurrently_create_copy more general

  19. Document the 'command' column of pg_stat_progress_repack

  20. Introduce the REPACK command

  21. Toggle logical decoding dynamically based on logical slot presence.

  22. Split vacuumdb to create vacuuming.c/h

  23. Remove ReorderBufferTupleBuf structure.

  24. Revert changes to CONCURRENTLY that "sped up" Xmin advance

  25. VACUUM: ignore indexing operations with CONCURRENTLY