Horton
Menu

Re: Adding REPACK [concurrently]

Antonin Houska <ah@cybertec.at>

View thread
From: Antonin Houska <ah@cybertec.at>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>, Srinath Reddy Sadipiralla <srinath2133@gmail.com>, Matthias van de Meent <boekewurm+postgres@gmail.com>, Pg Hackers <pgsql-hackers@lists.postgresql.org>, Robert Treat <rob@xzilla.net>
Date: 2026-03-25T09:00:49Z
Lists: pgsql-hackers 
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:

> - 0008 to 0010 are as posted by Antonin; they are unchanged, except for
>   fixes for the problems pointed out by Mihail.  Antonin, I would
>   appreciate it if you want to change the "reform" bit in 0007 as
>   discussed.

I've taken a look, but not sure if the tuple slots help here. In
heapam_relation_copy_for_cluster(), both table_scan_getnextslot() and
index_getnext_slot() call ExecStoreBufferHeapTuple() ->
tts_buffer_heap_store_tuple(), which AFAICS do not deform the tuple. Then
ExecFetchSlotHeapTuple() is used to retrieve the tuple, but again, the
underlying slot (TTSOpsBufferHeapTuple) handles it by copying rather than
deforming / forming. Thus I think the explicit "reforming" currently does not
add any performance overhead.

Of course, we can still use the slots, and do the following: 1) enforce tuple
deforming (by calling slot_getallattrs()), 2) set the dropped attributes to
NULL, 3) use ExecStoreVirtualTuple() to store the tuple into another slot and
4) get the heap tuple from the other slot. Should I do that? I'm asking
because I wasn't sure if you're concerned about performance or coding (or
both).


Whatever approach we take, I see two more opportunities for better
performance:

1. Do the "reforming" only if there are some dropped columns. (AFAICS even the
old CLUSTER / VACUUM FULL did not check this.)

2. Get rid of the values of dropped columns earlier, so that the dropped
values are not put into the tuplestore (likewise, I think that CLUSTER /
VACUUM FULL did not care.)


Besides that, I think that heap_form_tuple() should set the values of dropped
columns to NULL by default, or do I miss something? Anyway, this should be
addressed by a separate patch.

-- 
Antonin Houska
Web: https://www.cybertec-postgresql.com

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Allow old WAL recycling during REPACK CONCURRENTLY

  2. Advance restart_lsn more eagerly in LogicalConfirmReceivedLocation

  3. Remove unnecessary signal handler change

  4. Improve REPACK (CONCURRENTLY) error messages some more

  5. Revert "Allow logical replication snapshots to be database-specific"

  6. Move REPACK (CONCURRENTLY) test out of stock regression tests

  7. REPACK: do not require REPLICATION or LOGIN

  8. Add missing initialization

  9. Simplify declaration of memcpy target

  10. Reserve replication slots specifically for REPACK

  11. doc: Add an example of REPACK (CONCURRENTLY)

  12. Allow logical replication snapshots to be database-specific

  13. Avoid different-size pointer-to-integer cast

  14. Fix valgrind failure

  15. Add CONCURRENTLY option to REPACK

  16. Rename cluster.c to repack.c (and corresponding .h)

  17. Allow index_create to suppress index_build progress reporting

  18. Make index_concurrently_create_copy more general

  19. Document the 'command' column of pg_stat_progress_repack

  20. Introduce the REPACK command

  21. Toggle logical decoding dynamically based on logical slot presence.

  22. Split vacuumdb to create vacuuming.c/h

  23. Remove ReorderBufferTupleBuf structure.

  24. Revert changes to CONCURRENTLY that "sped up" Xmin advance

  25. VACUUM: ignore indexing operations with CONCURRENTLY