Horton
Menu

Re: CREATE INDEX CONCURRENTLY does not index prepared xact's data

Andrey Borodin <x4mmm@yandex-team.ru>

View thread
From: Andrey Borodin <x4mmm@yandex-team.ru>
To: Andres Freund <andres@anarazel.de>
Cc: Noah Misch <noah@leadboat.com>, PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>, Michael Paquier <michael@paquier.xyz>, Tom Lane <tgl@sss.pgh.pa.us>, Peter Geoghegan <pg@bowt.ie>
Date: 2021-08-29T18:38:03Z
Lists: pgsql-bugs 

> 29 авг. 2021 г., в 23:09, Andres Freund <andres@anarazel.de> написал(а):
>>> 
>>> It seems like it's going to add a substantial amount of work even when
>>> no 2PC xacts are involved?
>> Only if 2PCs are enabled.
> 
> I don't think that's good enough. Plenty of systems have 2PC enabled but very
> few if any transactions end up as 2PC ones.

I could not compose significantly better solution. Let's consider what we actually pay for.

GetLockConflicts() returns xids for 2PC and vxids for any other transaction. Then we wait for each other one-by-one.
GetLockConflicts() cannot return xids for regular transactions: these xids may not have xid lock yet. So we have to work with vxid.

Each wait case can resort to set of three possible actions: test/wait for xid (cheap), test/wait for vxid (relatively cheap), search xid by vxid (expensive).

When we have to wait for known-2PC we wait on xid (cheap).
When we have vxid:
1. If backend is not on the same vxid, we have to search xid by vxid (expensive). Then wait on obtained xid if any.
2. Either way wait for vxid, then for xid (cheap * 2).

So the really slow part is searching xids for long-gone vxids that may not have a xid at all.
We simply have no good mechanics to say that this vxid did not have a xid, if vxid is long gone.

Best optimisation I can imagine here is to gather all vxids with unknown xids and search for them in one call to TwoPhaseGetXidByVXid() with one LWLockAcquire(TwoPhaseStateLock, LW_SHARED).

Does it worth the complexity?

Best regards, Andrey Borodin.

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Improve contrib/amcheck's tests for CREATE INDEX CONCURRENTLY.

  2. Fix minor memory leaks in pg_dump.

  3. Avoid race in RelationBuildDesc() affecting CREATE INDEX CONCURRENTLY.

  4. Fix CREATE INDEX CONCURRENTLY for the newest prepared transactions.

  5. Add -w back to the flags for pg_ctl (re)start in PostgresNode

  6. Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transactions.