Horton

Re: Adding REPACK [concurrently]

Antonin Houska <ah@cybertec.at>

View thread
From: Antonin Houska <ah@cybertec.at>
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>
Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>, Fujii Masao <masao.fujii@gmail.com>, Robert Treat <rob@xzilla.net>, Pg Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2025-08-25T15:42:15Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Improve REPACK (CONCURRENTLY) error messages some more

  2. Add CONCURRENTLY option to REPACK

  3. Make index_concurrently_create_copy more general

  4. Document the 'command' column of pg_stat_progress_repack

  5. Introduce the REPACK command

  6. Split vacuumdb to create vacuuming.c/h

  7. Revert changes to CONCURRENTLY that "sped up" Xmin advance

Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:

> > Not sure I understand in all details, but I don't think SnapshotSelf is the
> > correct snapshot. Note that HeapTupleSatisfiesSelf() does not use its
> > 'snapshot' argument at all. Instead, it considers the set of running
> > transactions as it is at the time the function is called.
> 
> Yes, and it is almost the same behavior when a typical MVCC snapshot
> encounters a tuple created by its own transaction.
> 
> So, how it works in the non MVCC-safe case (current patch behaviour):
> 
> 1) we have a whole initial table snapshot with all the xmin = repack XID
> 2) appling transaction sees ALL the self-alive (no xmax) tuples in it
> because all tuples created\deleted by transaction itself
> 3) each update/delete during the replay selects the last existing
> tuple version, updates it xmax and inserts a new one
> 4) so, there is no any real MVCC involved - just find the latest
> version and create a new version
> 5) and it works correctly because all ordering issues were resolved by
> locking mechanisms on the original table or by reordering buffer

ok

> How it maps to MVCC-safe case (SnapshotSelf):
> 
> 1) we have a whole initial table snapshot with all xmin copied from
> the original table. All such xmin are committed.
> 2) appling transaction sees ALL the self-alive (no xmax) tuple in it
> because its xmin\xmax is committed and SnapshotSelf is happy with it

How does HeapTupleSatisfiesSelf() recognize the status of any XID w/o using a
snapshot? Do you mean by checking the commit log (TransactionIdDidCommit) ?

> 3) each update/delete during the replay selects the last existing
> tuple version, updates it xmax=original xid and inserts a new one
> keeping with xmin=orignal xid
> 4) --//--
> 5) --//--

> > However, at the time we're replaying the UPDATE in the new table, the tuple
> > may have been already deleted from the old table, and the deleting transaction
> > may already have committed. In such a case, HeapTupleSatisfiesSelf() will
> > conclude the old version invisible and the we'll fail to replay the UPDATE.
> 
> No, it will see it - because its xmax will be empty in the repacked
> version of the table.

You're right, it'll be empty in the new table.

-- 
Antonin Houska
Web: https://www.cybertec-postgresql.com