Re: GiST VACUUM

x4mmm@yandex-team.ru

From: Andrey Borodin <x4mmm@yandex-team.ru>
To: Heikki Linnakangas <hlinnaka@iki.fi>
Cc: Michael Paquier <michael@paquier.xyz>, Thomas Munro <thomas.munro@enterprisedb.com>, Robert Haas <robertmhaas@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, Костя Кузнецов <chapaev28@ya.ru>
Date: 2019-03-04T18:26:27Z
Lists: pgsql-hackers
Hi!

Thanks for fixing gist amcheck! The idea of using these two patches together seems so obvious now, but never actually visited my mind before.

> 4 марта 2019 г., в 18:04, Heikki Linnakangas <hlinnaka@iki.fi> написал(а):
> Thanks! As I noted at https://www.postgresql.org/message-id/2ff57b1f-01b4-eacf-36a2-485a12017f6e%40iki.fi, the test patch left the index corrupt. I fixed it so that it leaves behind incompletely split pages, without the corruption, see attached. It's similar to yours, but let me recap what it does:
> 
> * Hacks gistbuild(), create 100 empty pages immediately after the root pages. They are leaked, so they won't be reused until the a VACUUM sees them and puts them to the FSM
> 
> * Hacks gistinserttuples(), to leave the split incompleted with 50% probability
> 
> * In vacuum, print a line to the log whenever it needs to "jump left"
> 
> I used this patch, with the attached test script that's similar to yours, but it also tries to verify that the index returns correct results. It prints a result set like this:
> 
>   sum
> ---------
> -364450
>  364450
> (2 rows)
> 
> If the two numbers add up to 0, the index seems valid. And you should see "RESCAN" lines in the log, to show that jumping left happened. Because the behavior is random and racy, you may need to run the script many times to see both "RESCAN TRIGGERED BY NSN" and "RESCAN TRIGGERED BY FollowRight" cases. Especially the "FollowRight" case happens less frequently than the NSN case, you might need to run the script > 10 times to see it.
Great! I've repeated your tests on my machine, I observe similar frequencies of causes of rescan left jumps.

> I also tried your amcheck tool with this. It did not report any errors.
> 
> Attached is also latest version of the patch itself. It is the same as your latest patch v19, except for some tiny comment kibitzing. I'll mark this as Ready for Committer in the commitfest app, and will try to commit it in the next couple of days.

That's cool! I'll work on 2nd step of these patchset to make blockset data structure prettier and less hacky.

Best regards, Andrey Borodin.

Commits

  1. Use full 64-bit XID for checking if a deleted GiST page is old enough.

  2. Refactor checks for deleted GiST pages.

  3. Delete empty pages during GiST VACUUM.

  4. Scan GiST indexes in physical order during VACUUM.

  5. Prevent GIN deleted pages from being reclaimed too early