Horton
Menu

Re: Adding REPACK [concurrently]

Mihail Nikalayeu <mihailnikalayeu@gmail.com>

View thread
From: Mihail Nikalayeu <mihailnikalayeu@gmail.com>
To: Antonin Houska <ah@cybertec.at>
Cc: Andres Freund <andres@anarazel.de>, Amit Kapila <amit.kapila16@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Srinath Reddy Sadipiralla <srinath2133@gmail.com>, Matthias van de Meent <boekewurm+postgres@gmail.com>, Pg Hackers <pgsql-hackers@lists.postgresql.org>, Robert Treat <rob@xzilla.net>
Date: 2026-04-17T02:01:00Z
Lists: pgsql-hackers

Attachments

Hello!

I think I got working POC for deadlock-detector enhancements for
REPACK (and potentially other).

Each PGPROC gets one FutureWaitLock slot: a (locktag, mode) pair,
which is empty when locktag_lockmethodid == 0.
A backend calls LockDeclareFutureWait() to publish its intent. Only
REPACK (CONCURRENTLY) uses it today, declaring its future
AccessExclusiveLock.

Only the owning backend writes its slot; it does so under the
partition lock covering the declared tag. Remote readers (the deadlock
detector) must hold that partition lock to observe it.
The detector holds all partition locks during its graph walk, so it
can read any slot safely. The slot clears in LockAcquireExtended() at
the exact point the backend attaches its waitLink for the same tag and
mode.
It is also cleared on abort, backend exit, and end of transaction.

FindLockCycleRecurseFuture() treats the declared future lock as a hard
edge to (a) current holders whose mode conflicts and (b)
already-queued waiters that JoinWaitQueue() would place ahead of the
future request.

Weak relation locks are kept in per-backend fast-path arrays and
remain invisible to the detector's graph walk.
Normally LockAcquireExtended() migrates them to the main lock table
when a conflicting strong lock is requested.
This action is gated by FastPathStrongRelationLocks->count to prevent
new fast-path lock processing.

A future-wait declaration does not trigger that migration due to
performance considerations.

Instead CheckDeadLock() now snapshots currently declared
fast-path-incompatible future waits and increments
FastPathStrongRelationLocks->count for those tags (blocking new
fast-path acquisitions).
It then calls FastPathTransferRelationLocks() for each such tag to
move existing holders into the main table, making everything visible
to the deadlock walker.
Once the deadlock checker finishes, FastPathStrongRelationLocks->count
is decremented enabling fast-path processing again. Performance
degradation occurs only during deadlock processing and only if REPACK
is active.

If a deadlock contains a "future" edge it is reported as "future
deadlock detected".

Repack first declares future AE and only then proceeds to actually get SUE.

Added test cases for different scenarios, including SUE from another
backend and a 3-backend deadlock.

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Allow old WAL recycling during REPACK CONCURRENTLY

  2. Advance restart_lsn more eagerly in LogicalConfirmReceivedLocation

  3. Remove unnecessary signal handler change

  4. Improve REPACK (CONCURRENTLY) error messages some more

  5. Revert "Allow logical replication snapshots to be database-specific"

  6. Move REPACK (CONCURRENTLY) test out of stock regression tests

  7. REPACK: do not require REPLICATION or LOGIN

  8. Add missing initialization

  9. Simplify declaration of memcpy target

  10. Reserve replication slots specifically for REPACK

  11. doc: Add an example of REPACK (CONCURRENTLY)

  12. Allow logical replication snapshots to be database-specific

  13. Avoid different-size pointer-to-integer cast

  14. Fix valgrind failure

  15. Add CONCURRENTLY option to REPACK

  16. Rename cluster.c to repack.c (and corresponding .h)

  17. Allow index_create to suppress index_build progress reporting

  18. Make index_concurrently_create_copy more general

  19. Document the 'command' column of pg_stat_progress_repack

  20. Introduce the REPACK command

  21. Toggle logical decoding dynamically based on logical slot presence.

  22. Split vacuumdb to create vacuuming.c/h

  23. Remove ReorderBufferTupleBuf structure.

  24. Revert changes to CONCURRENTLY that "sped up" Xmin advance

  25. VACUUM: ignore indexing operations with CONCURRENTLY