Horton
Menu

Re: Adding REPACK [concurrently]

Antonin Houska <ah@cybertec.at>

View thread
From: Antonin Houska <ah@cybertec.at>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>, Srinath Reddy Sadipiralla <srinath2133@gmail.com>, Matthias van de Meent <boekewurm+postgres@gmail.com>, Pg Hackers <pgsql-hackers@lists.postgresql.org>, Robert Treat <rob@xzilla.net>
Date: 2026-03-25T16:52:56Z
Lists: pgsql-hackers 
Antonin Houska <ah@cybertec.at> wrote:

> Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> 
> > - 0008 to 0010 are as posted by Antonin; they are unchanged, except for
> >   fixes for the problems pointed out by Mihail.  Antonin, I would
> >   appreciate it if you want to change the "reform" bit in 0007 as
> >   discussed.
> 
> I've taken a look, but not sure if the tuple slots help here. In
> heapam_relation_copy_for_cluster(), both table_scan_getnextslot() and
> index_getnext_slot() call ExecStoreBufferHeapTuple() ->
> tts_buffer_heap_store_tuple(), which AFAICS do not deform the tuple. Then
> ExecFetchSlotHeapTuple() is used to retrieve the tuple, but again, the
> underlying slot (TTSOpsBufferHeapTuple) handles it by copying rather than
> deforming / forming. Thus I think the explicit "reforming" currently does not
> add any performance overhead.

Well, the deform / form steps do add some overhead of course, but these are
necessary to get rid of the values of the dropped columns. I wanted to say
that it wouldn't be cheaper with slots, because then we'd have to enforce the
deform / form steps too, although the coding would be different:

> Of course, we can still use the slots, and do the following: 1) enforce tuple
> deforming (by calling slot_getallattrs()), 2) set the dropped attributes to
> NULL, 3) use ExecStoreVirtualTuple() to store the tuple into another slot and
> 4) get the heap tuple from the other slot. Should I do that? I'm asking
> because I wasn't sure if you're concerned about performance or coding (or
> both).

> Whatever approach we take, I see two more opportunities for better
> performance:
> 
> 1. Do the "reforming" only if there are some dropped columns. (AFAICS even the
> old CLUSTER / VACUUM FULL did not check this.)

I think this would need more work because CLUSTER / VACCUM FULL / REPACK do
not remove the dropped attributes from the tuple descriptor. So the
optimization would only work until the first column is dropped. All the
following runs would then do the reforming even if no other colmns were droped
since the previous run.

Perhaps we can teach REPACK to remove dropped columns from the tuple
descriptor in the future.

-- 
Antonin Houska
Web: https://www.cybertec-postgresql.com

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Allow old WAL recycling during REPACK CONCURRENTLY

  2. Advance restart_lsn more eagerly in LogicalConfirmReceivedLocation

  3. Remove unnecessary signal handler change

  4. Improve REPACK (CONCURRENTLY) error messages some more

  5. Revert "Allow logical replication snapshots to be database-specific"

  6. Move REPACK (CONCURRENTLY) test out of stock regression tests

  7. REPACK: do not require REPLICATION or LOGIN

  8. Add missing initialization

  9. Simplify declaration of memcpy target

  10. Reserve replication slots specifically for REPACK

  11. doc: Add an example of REPACK (CONCURRENTLY)

  12. Allow logical replication snapshots to be database-specific

  13. Avoid different-size pointer-to-integer cast

  14. Fix valgrind failure

  15. Add CONCURRENTLY option to REPACK

  16. Rename cluster.c to repack.c (and corresponding .h)

  17. Allow index_create to suppress index_build progress reporting

  18. Make index_concurrently_create_copy more general

  19. Document the 'command' column of pg_stat_progress_repack

  20. Introduce the REPACK command

  21. Toggle logical decoding dynamically based on logical slot presence.

  22. Split vacuumdb to create vacuuming.c/h

  23. Remove ReorderBufferTupleBuf structure.

  24. Revert changes to CONCURRENTLY that "sped up" Xmin advance

  25. VACUUM: ignore indexing operations with CONCURRENTLY