Re: Adding REPACK [concurrently]

Antonin Houska <ah@cybertec.at>

From: Antonin Houska <ah@cybertec.at>
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>
Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>, Pg Hackers <pgsql-hackers@lists.postgresql.org>, Robert Treat <rob@xzilla.net>
Date: 2025-12-02T16:14:57Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. Improve REPACK (CONCURRENTLY) error messages some more

  2. Add CONCURRENTLY option to REPACK

  3. Make index_concurrently_create_copy more general

  4. Document the 'command' column of pg_stat_progress_repack

  5. Introduce the REPACK command

  6. Split vacuumdb to create vacuuming.c/h

  7. Revert changes to CONCURRENTLY that "sped up" Xmin advance

Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:

> Hello, Antonin!
> 
> On Mon, Nov 3, 2025 at 8:56 AM Antonin Houska <ah@cybertec.at> wrote:
> > I'll fix all the problems in the next version. Thanks!
> 
> A few more moments I mentioned:
> 
> > switch ((vis = HeapTupleSatisfiesVacuum(tuple, OldestXmin, buf)))
> vis is unused, also to double braces.
> 
> >       LockBuffer(buf, BUFFER_LOCK_UNLOCK);
> >       continue;
> >    }
> 
> >    /*
> >     * In the concurrent case, we have a copy of the tuple, so we
> >     * don't worry whether the source tuple will be deleted / updated
> >     * after we release the lock.
> >     */
> >    LockBuffer(buf, BUFFER_LOCK_UNLOCK);
> >}
> 
> I think locking and comments are a little bit confusing here.
> I think we may use single LockBuffer(buf, BUFFER_LOCK_UNLOCK); before
> `if (isdead)` as it was before.
> Also, I am not sure "we have a copy" is the correct point here, I
> think motivation is mostly the same as in
> heapam_index_build_range_scan.

All these problems are due to incorrect separation of the "preserve
visibility" part of the patch series. Will be fixed in the next version.

> Also, I think it is a good idea to add tests for index-based and
> sort-based repack.

Not sure, cluster.sql already seems to do the same.

> Also, for sort-based I think we need to also call
> repack_decode_concurrent_changes during insertion phase

I miss the point. The current coding is such that this part

	if (concurrent)
	{
		XLogRecPtr	end_of_wal;

		end_of_wal = GetFlushRecPtr(NULL);
		if ((end_of_wal - end_of_wal_prev) > wal_segment_size)
		{
			repack_decode_concurrent_changes(decoding_ctx, end_of_wal);
			end_of_wal_prev = end_of_wal;
		}
	}

gets called regardless the value of 'tuplesort' above.

> > is_system_catalog && !concurrent
> 2 places, always true, feels strange.

ok

-- 
Antonin Houska
Web: https://www.cybertec-postgresql.com