Re: GiST VACUUM

Heikki Linnakangas <hlinnaka@iki.fi>

From: Heikki Linnakangas <hlinnaka@iki.fi>
To: Andrey Borodin <x4mmm@yandex-team.ru>
Cc: Michael Paquier <michael@paquier.xyz>, Thomas Munro <thomas.munro@enterprisedb.com>, Robert Haas <robertmhaas@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, Костя Кузнецов <chapaev28@ya.ru>
Date: 2019-03-04T10:27:04Z
Lists: pgsql-hackers
On 04/01/2019 21:26, Andrey Borodin wrote:
> 3 янв. 2019 г., в 23:47, Andrey Borodin <x4mmm@yandex-team.ru>
> написал(а):
>> 
>>> * Bitmapset stores 32 bit signed integers, but BlockNumber is
>>> unsigned. So this will fail with an index larger than 2^31
>>> blocks. That's perhaps academical, I don't think anyone will try
>>> to create a 16 TB GiST index any time soon. But it feels wrong to
>>> introduce an arbitrary limitation like that.
>> 
>> Looks like bitmapset is unsuitable for collecting block numbers due
>> to the interface. Let's just create custom container for this? Or
>> there is one other option: for each block number throw away sign
>> bit and consider page potentially internal and potentially empty
>> leaf if (blkno & 0x7FFFFFF) is in bitmapset. And then we will have
>> to create at least one 17Tb GiST to check it actually works.
> 
> Heikki, how do you think, is implementing our own radix tree for this
> is viable solution? I've written working implementation with 4-level
> statically typed tree. If we follow this route, probably, there must
> be tests for this data structure.

Yeah, seems reasonable.

- Heikki


Commits

  1. Use full 64-bit XID for checking if a deleted GiST page is old enough.

  2. Refactor checks for deleted GiST pages.

  3. Delete empty pages during GiST VACUUM.

  4. Scan GiST indexes in physical order during VACUUM.

  5. Prevent GIN deleted pages from being reclaimed too early