Horton
Menu

Re: Adding REPACK [concurrently]

Alvaro Herrera <alvherre@alvh.no-ip.org>

View thread
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
To: Antonin Houska <ah@cybertec.at>
Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>, Srinath Reddy Sadipiralla <srinath2133@gmail.com>, Matthias van de Meent <boekewurm+postgres@gmail.com>, Pg Hackers <pgsql-hackers@lists.postgresql.org>, Robert Treat <rob@xzilla.net>
Date: 2026-03-26T17:28:12Z
Lists: pgsql-hackers 
On 2026-Mar-25, Antonin Houska wrote:

> I've taken a look, but not sure if the tuple slots help here. In
> heapam_relation_copy_for_cluster(), both table_scan_getnextslot() and
> index_getnext_slot() call ExecStoreBufferHeapTuple() ->
> tts_buffer_heap_store_tuple(), which AFAICS do not deform the tuple.
> Then ExecFetchSlotHeapTuple() is used to retrieve the tuple, but
> again, the underlying slot (TTSOpsBufferHeapTuple) handles it by
> copying rather than deforming / forming. Thus I think the explicit
> "reforming" currently does not add any performance overhead.

Looking again, I think we should leave this alone for now.  The existing
code (looking at heapam_relation_copy_for_cluster) is somewhat silly, in
that we do a bunch of operations with slots, then we heap-ify the tuple
by doing ExecFetchSlotHeapTuple(), then we do the reform_and_rewrite
dance, which must deform the tuple only to immediately form it back; and
after we've done that, we make it go through the rewriteheap.c motions,
which again deforms and forms back (in certain cases).

This code structure seems mostly a relic from the time when heapam.c was
all we had, and I think we should rewrite it from scratch to work using
only slots.  But not for this patch, and not for pg19, because I think
that's going to take some nontrivial effort on its own.  So for REPACK
in pg19 I think we should just do the simplest thing possible, and do
the minimum amount of deform/form dance necessary to make this whole
thing work.  I suppose that that's what the submitted code (v44-0008)
does, so I'm going to leave it at that.

-- 
Álvaro Herrera               48°01'N 7°57'E  —  https://www.EnterpriseDB.com/
"In Europe they call me Niklaus Wirth; in the US they call me Nickel's worth.
 That's because in Europe they call me by name, and in the US by value!"

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Allow old WAL recycling during REPACK CONCURRENTLY

  2. Advance restart_lsn more eagerly in LogicalConfirmReceivedLocation

  3. Remove unnecessary signal handler change

  4. Improve REPACK (CONCURRENTLY) error messages some more

  5. Revert "Allow logical replication snapshots to be database-specific"

  6. Move REPACK (CONCURRENTLY) test out of stock regression tests

  7. REPACK: do not require REPLICATION or LOGIN

  8. Add missing initialization

  9. Simplify declaration of memcpy target

  10. Reserve replication slots specifically for REPACK

  11. doc: Add an example of REPACK (CONCURRENTLY)

  12. Allow logical replication snapshots to be database-specific

  13. Avoid different-size pointer-to-integer cast

  14. Fix valgrind failure

  15. Add CONCURRENTLY option to REPACK

  16. Rename cluster.c to repack.c (and corresponding .h)

  17. Allow index_create to suppress index_build progress reporting

  18. Make index_concurrently_create_copy more general

  19. Document the 'command' column of pg_stat_progress_repack

  20. Introduce the REPACK command

  21. Toggle logical decoding dynamically based on logical slot presence.

  22. Split vacuumdb to create vacuuming.c/h

  23. Remove ReorderBufferTupleBuf structure.

  24. Revert changes to CONCURRENTLY that "sped up" Xmin advance

  25. VACUUM: ignore indexing operations with CONCURRENTLY