Horton
Menu

Re: Adding REPACK [concurrently]

Mihail Nikalayeu <mihailnikalayeu@gmail.com>

View thread
From: Mihail Nikalayeu <mihailnikalayeu@gmail.com>
To: Antonin Houska <ah@cybertec.at>
Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>, Pg Hackers <pgsql-hackers@lists.postgresql.org>, Robert Treat <rob@xzilla.net>
Date: 2026-02-01T19:46:31Z
Lists: pgsql-hackers

Attachments

Hello!


PART 1:

--------------

Something still wrong with 0006, check:

'pgbench: error: client 12 script 0 aborted in command 2 query 0: ERROR:
 attempted to overwrite invisible tuple
https://cirrus-ci.com/task/6385612527239168?logs=test_world#L300

But it is hard to reproduce - happened once.

--------------

Also, once I got
[16:25:18.641] #   at /tmp/cirrus-ci-build/contrib/amcheck/t/
007_repack_concurrently.pl line 57.
[16:25:18.641] #                   'pgbench: error: client 6 script 0
aborted in command 2 query 0: ERROR:  relation 21856 deleted while still in
use
https://cirrus-ci.com/task/4686014242881536?logs=test_world#L384

It was the PROC_IN_REPACK version (see below), but I think it is not
related to it. But I'm not 100% sure.


PART 2:

> I'm considering a special kind of relation whose catalog entries remain
in the
>  catalog cache and are never written to the catalog tables. (Unlike
temporary
>  relation, it'd be WAL logged so that REPACK can be replayed on standby.)

I think it is too complicated, especially including replication logic.
Approach with catalog-only xid is much simpler, it was even committed (yes,
reverted but because of another reason).

Essentially we have two issues:
1) make sure catalog entities are not dropped because the vacuum
2) make sure data in new table is not vacuumed also

For the first PROC_IN_REPACK is enough.
For second - depends if MVCC-safe (original xmin/xmax) are preserved. If
yes - looks like nothing more needed.

If not - just prevent the vacuum from touching the table (but, looks like
it is done already, because lock is held on NewHeap until commit).
And additionally reset snapshots during the index building itself, but it
is scope of another patch.

I have implemented PROC_IN_REPACK POC  in the attached patch.

Also, I am still not sure if MVCC-safe implementation is worth
its complexity compared with "relcheckxmin"approach [0].

[0]:
https://www.postgresql.org/message-id/CADzfLwUEH5%2BLjCN%2B6kRfSsXwuou8rKXyVV42Wi-O_TG0360Kug%40mail.gmail.com

Best regards,
Mikhail.

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Allow old WAL recycling during REPACK CONCURRENTLY

  2. Advance restart_lsn more eagerly in LogicalConfirmReceivedLocation

  3. Remove unnecessary signal handler change

  4. Improve REPACK (CONCURRENTLY) error messages some more

  5. Revert "Allow logical replication snapshots to be database-specific"

  6. Move REPACK (CONCURRENTLY) test out of stock regression tests

  7. REPACK: do not require REPLICATION or LOGIN

  8. Add missing initialization

  9. Simplify declaration of memcpy target

  10. Reserve replication slots specifically for REPACK

  11. doc: Add an example of REPACK (CONCURRENTLY)

  12. Allow logical replication snapshots to be database-specific

  13. Avoid different-size pointer-to-integer cast

  14. Fix valgrind failure

  15. Add CONCURRENTLY option to REPACK

  16. Rename cluster.c to repack.c (and corresponding .h)

  17. Allow index_create to suppress index_build progress reporting

  18. Make index_concurrently_create_copy more general

  19. Document the 'command' column of pg_stat_progress_repack

  20. Introduce the REPACK command

  21. Toggle logical decoding dynamically based on logical slot presence.

  22. Split vacuumdb to create vacuuming.c/h

  23. Remove ReorderBufferTupleBuf structure.

  24. Revert changes to CONCURRENTLY that "sped up" Xmin advance

  25. VACUUM: ignore indexing operations with CONCURRENTLY