Horton
Menu

Re: Adding REPACK [concurrently]

Alvaro Herrera <alvherre@alvh.no-ip.org>

View thread
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
To: Antonin Houska <ah@cybertec.at>
Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>, Amit Kapila <amit.kapila16@gmail.com>, Mihail Nikalayeu <mihailnikalayeu@gmail.com>, Matthias van de Meent <boekewurm+postgres@gmail.com>, Pg Hackers <pgsql-hackers@lists.postgresql.org>, Robert Treat <rob@xzilla.net>
Date: 2026-04-03T14:59:09Z
Lists: pgsql-hackers

Attachments

On 2026-Apr-03, Antonin Houska wrote:

> This is an alternative implementation of 0006, allowing one backend running
> REPACK (CONCURRENTLY) in a database, instead of one backend in a cluster.

Thanks!  so I'm removing the previous one and taking this one.  Here's a
v50:

- In testing, I noticed that we could sometimes request a Flush for a
  WAL position that hasn't been written yet.  This is due to my
  replacing the original code that wrote a dummy xlog record that we
  could flush, with a call to XLogGetInsertRecPtr().  So we'd get an
  error like

LOG:  request to flush past end of generated WAL; request 0/15CF0018, current position 0/15CF000
  Antonin promptly noticed that this is because XLogGetInsertRecPtr()
  gets the LSN past the segment header, which is 18 bytes wrong.  So the
  fix here is to use XLogGetInsertEndRecPtr() instead.
  
- My testing also uncovered a problem with exclusion constraints; tables
  with them would fail to repack like
  ERROR:  exclusion constraint record missing for rel temporal_fk_mltrng2mltrng_pk_repacknew
  Antonin sent a patch to create copies of the constraints on the
  transient index, which seems like it fixes the problem.  Those
  constraints are obviously discarded together with the transient index.

- I polished the patch to reserve replication slots for REPACK.  Given
  the new implementation of 0006 that was submitted implies that we can
  now run multiple repacks concurrently, I changed the default of 1 to 5.

-- 
Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/
"Here's a general engineering tip: if the non-fun part is too complex for you
to figure out, that might indicate the fun part is too ambitious." (John Naylor)
https://postgr.es/m/CAFBsxsG4OWHBbSDM%3DsSeXrQGOtkPiOEOuME4yD7Ce41NtaAD9g%40mail.gmail.com

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Allow old WAL recycling during REPACK CONCURRENTLY

  2. Advance restart_lsn more eagerly in LogicalConfirmReceivedLocation

  3. Remove unnecessary signal handler change

  4. Improve REPACK (CONCURRENTLY) error messages some more

  5. Revert "Allow logical replication snapshots to be database-specific"

  6. Move REPACK (CONCURRENTLY) test out of stock regression tests

  7. REPACK: do not require REPLICATION or LOGIN

  8. Add missing initialization

  9. Simplify declaration of memcpy target

  10. Reserve replication slots specifically for REPACK

  11. doc: Add an example of REPACK (CONCURRENTLY)

  12. Allow logical replication snapshots to be database-specific

  13. Avoid different-size pointer-to-integer cast

  14. Fix valgrind failure

  15. Add CONCURRENTLY option to REPACK

  16. Rename cluster.c to repack.c (and corresponding .h)

  17. Allow index_create to suppress index_build progress reporting

  18. Make index_concurrently_create_copy more general

  19. Document the 'command' column of pg_stat_progress_repack

  20. Introduce the REPACK command

  21. Toggle logical decoding dynamically based on logical slot presence.

  22. Split vacuumdb to create vacuuming.c/h

  23. Remove ReorderBufferTupleBuf structure.

  24. Revert changes to CONCURRENTLY that "sped up" Xmin advance

  25. VACUUM: ignore indexing operations with CONCURRENTLY