Re: GiST VACUUM

Heikki Linnakangas <hlinnaka@iki.fi>

From: Heikki Linnakangas <hlinnaka@iki.fi>
To: Andrey Borodin <x4mmm@yandex-team.ru>
Cc: Thomas Munro <thomas.munro@gmail.com>, Michael Paquier <michael@paquier.xyz>, Thomas Munro <thomas.munro@enterprisedb.com>, Robert Haas <robertmhaas@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, Костя Кузнецов <chapaev28@ya.ru>
Date: 2019-06-27T21:19:15Z
Lists: pgsql-hackers
On 27/06/2019 20:15, Andrey Borodin wrote:
> But I have stupid question again, about this code:
> 
> https://github.com/x4m/postgres_g/commit/096d5586537d29ff316ca8ce074bbe1b325879ee#diff-754126824470cb8e68fd5e32af6d3bcaR417
> 
> 		nextFullXid = ReadNextFullTransactionId();
> 			diff = U64FromFullTransactionId(nextFullXid) -
> 				U64FromFullTransactionId(latestRemovedFullXid);
> 			if (diff < MaxTransactionId / 2)
> 			{
> 				TransactionId latestRemovedXid;
> 					
> 				// sleep(100500 hours); latestRemovedXid becomes xid from future
> 	
> 				latestRemovedXid = XidFromFullTransactionId(latestRemovedFullXid);
> 				ResolveRecoveryConflictWithSnapshot(latestRemovedXid,
> 													xlrec->node);
> 			}
> 
> Do we have a race condition here? Can latestRemovedXid overlap and start to be xid from future?
> I understand that it is purely hypothetical, but still latestRemovedXid is from ancient past already.

Good question. No, that can't happen, because this code is in the WAL 
redo function. In a standby, the next XID counter only moves forward 
when a WAL record is replayed that advances it, and all WAL records are 
replayed serially, so that can't happen when we're in the middle of 
replaying this record. A comment on that would be good, though.

When I originally had the check like above in the code that created the 
WAL record, it had exactly that problem, because in the master the next 
XID counter can advance concurrently.

- Heikki



Commits

  1. Use full 64-bit XID for checking if a deleted GiST page is old enough.

  2. Refactor checks for deleted GiST pages.

  3. Delete empty pages during GiST VACUUM.

  4. Scan GiST indexes in physical order during VACUUM.

  5. Prevent GIN deleted pages from being reclaimed too early