Horton
Menu

Re: Adding REPACK [concurrently]

Antonin Houska <ah@cybertec.at>

View thread
From: Antonin Houska <ah@cybertec.at>
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>
Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>, Pg Hackers <pgsql-hackers@lists.postgresql.org>, Robert Treat <rob@xzilla.net>
Date: 2026-02-06T16:29:58Z
Lists: pgsql-hackers 
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:

> > I think it *is* related. My earlier patch version, which used the
> > PROC_IN_VACUUM flag improperly [1] was also causing visibility issues.  Please
> > let me know if you manage to reproduce the issue with v32.
> 
> Will try. Just to highlight - first error happened on v31 *without* PROC_IN_REPACK.

I spent some time running the test with your branch (based on v32 as you told
me off-list), but couldn't reproduce the problem.

The related code is in heap_inplace_lock():

    /* no known way this can happen */
    ereport(ERROR,
		    (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
		     errmsg_internal("attempted to overwrite invisible tuple")));


The only path REPACK (CONCURRENTLY) uses in-place update seems to be:

cluster.c:build_new_indexes() -> index_create_copy() -> index_create() ->
index_build() -> index_update_stats() -> systable_inplace_update_begin()


However I've got no idea how this can be related to REPACK. Since the new
index is not visible to other transactions until REPACK is done, VACUUM should
be the only process able to change the tuple before
heap_inplace_lock(). Indeed, the server log seems to indicate relationship to
VACUUM:

2026-02-01 16:44:58.878 UTC autovacuum worker[22589] LOG:  automatic vacuum of table "postgres.pg_catalog.pg_class": index scans: 1
...
2026-02-01 16:44:58.884 UTC client backend[12737] 008_repack_concurrently.pl LOG:  statement: COMMIT;
2026-02-01 16:44:58.884 UTC client backend[12737] 008_repack_concurrently.pl LOG:  statement: SELECT pg_try_advisory_lock(42)::integer AS gotlock 
2026-02-01 16:44:58.884 UTC client backend[12737] 008_repack_concurrently.pl LOG:  statement: SELECT pg_advisory_lock(43);
2026-02-01 16:44:58.884 UTC client backend[12737] 008_repack_concurrently.pl LOG:  statement: BEGIN;
2026-02-01 16:44:58.884 UTC client backend[12737] 008_repack_concurrently.pl LOG:  statement: INSERT INTO tbl(j) VALUES (nextval('last_j')) RETURNING j 
2026-02-01 16:44:58.885 UTC client backend[12727] 008_repack_concurrently.pl LOG:  statement: SELECT COUNT(*) AS count FROM tbl WHERE j <= 14148 
2026-02-01 16:44:58.885 UTC client backend[12734] 008_repack_concurrently.pl LOG:  statement: SELECT COUNT(*) AS count FROM tbl WHERE j <= 14145 
2026-02-01 16:44:58.885 UTC client backend[12737] 008_repack_concurrently.pl LOG:  statement: COMMIT;
2026-02-01 16:44:58.885 UTC client backend[12737] 008_repack_concurrently.pl LOG:  statement: SELECT pg_advisory_unlock(43);
2026-02-01 16:44:58.887 UTC client backend[12737] 008_repack_concurrently.pl LOG:  statement: BEGIN
				--TRANSACTION ISOLATION LEVEL REPEATABLE READ
				;
2026-02-01 16:44:58.887 UTC client backend[12737] 008_repack_concurrently.pl LOG:  statement: SELECT 1;
2026-02-01 16:44:58.891 UTC REPACK decoding worker[22621] FATAL:  terminating background worker "REPACK decoding worker" due to administrator command
2026-02-01 16:44:58.896 UTC client backend[12740] 008_repack_concurrently.pl LOG:  statement: SELECT COUNT(*) AS count FROM tbl WHERE j <= 14146 
2026-02-01 16:44:58.896 UTC client backend[12722] 008_repack_concurrently.pl ERROR:  attempted to overwrite invisible tuple
2026-02-01 16:44:58.896 UTC client backend[12722] 008_repack_concurrently.pl STATEMENT:  REPACK (CONCURRENTLY) tbl USING INDEX tbl_pkey;


However, VACUUM should not touch the tuple because the scan in
systable_inplace_update_begin() should leave the containing buffer pinned. I
wonder if you managed to hit another existing bug.

-- 
Antonin Houska
Web: https://www.cybertec-postgresql.com

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Allow old WAL recycling during REPACK CONCURRENTLY

  2. Advance restart_lsn more eagerly in LogicalConfirmReceivedLocation

  3. Remove unnecessary signal handler change

  4. Improve REPACK (CONCURRENTLY) error messages some more

  5. Revert "Allow logical replication snapshots to be database-specific"

  6. Move REPACK (CONCURRENTLY) test out of stock regression tests

  7. REPACK: do not require REPLICATION or LOGIN

  8. Add missing initialization

  9. Simplify declaration of memcpy target

  10. Reserve replication slots specifically for REPACK

  11. doc: Add an example of REPACK (CONCURRENTLY)

  12. Allow logical replication snapshots to be database-specific

  13. Avoid different-size pointer-to-integer cast

  14. Fix valgrind failure

  15. Add CONCURRENTLY option to REPACK

  16. Rename cluster.c to repack.c (and corresponding .h)

  17. Allow index_create to suppress index_build progress reporting

  18. Make index_concurrently_create_copy more general

  19. Document the 'command' column of pg_stat_progress_repack

  20. Introduce the REPACK command

  21. Toggle logical decoding dynamically based on logical slot presence.

  22. Split vacuumdb to create vacuuming.c/h

  23. Remove ReorderBufferTupleBuf structure.

  24. Revert changes to CONCURRENTLY that "sped up" Xmin advance

  25. VACUUM: ignore indexing operations with CONCURRENTLY