Horton
Menu

Re: 64-bit XIDs in deleted nbtree pages

Masahiko Sawada <sawada.mshk@gmail.com>

View thread
From: Masahiko Sawada <sawada.mshk@gmail.com>
To: Peter Geoghegan <pg@bowt.ie>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Heikki Linnakangas <hlinnaka@iki.fi>
Date: 2021-02-19T06:18:08Z
Lists: pgsql-hackers 
On Fri, Feb 19, 2021 at 3:12 PM Peter Geoghegan <pg@bowt.ie> wrote:
>
> On Thu, Feb 18, 2021 at 3:13 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> > Agreed. Thanks for your explanation.
>
> Attached is v5, which has some of the changes I talked about. Changes
> from v4 include:
>
> * Now only updates metapage during btvacuumcleanup() in the first
> patch, which is enough to fix the existing
> IndexVacuumInfo.num_heap_tuples issue.
>
> * Restored _bt_getbuf() page-from-FSM XID check. Out of sheer paranoia.
>
> * The second patch in the series now respects work_mem when sizing the
> BTPendingRecycle array.
>
> * New enhancement to the XID GlobalVisCheckRemovableFullXid() test
> used in the second patch, to allow it to recycle even more pages.
> (Still unsure of some of the details here.)

Thank you for updating the patch!

>
> I would like to commit the first patch in a few days -- I refer to the
> big patch that makes deleted page XIDs 64-bit/full. Can you take a
> look at that one, Masahiko? That would be helpful. I can produce a bug
> fix for the IndexVacuumInfo.num_heap_tuples issue fairly easily, but I
> think that that should be written after the first patch is finalized
> and committed.

I'll look at the first patch first.

>
> The second patch (the new recycling optimization) will require more
> work and testing.

Then also look at those patches.

Regards,

-- 
Masahiko Sawada
EDB:  https://www.enterprisedb.com/

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Recycle nbtree pages deleted during same VACUUM.

  2. VACUUM VERBOSE: Count "newly deleted" index pages.

  3. Use full 64-bit XIDs in deleted nbtree pages.

  4. Fix nbtree cleanup-only VACUUM stats inaccuracies.

  5. Fix undercounting in VACUUM VERBOSE output.

  6. Delete empty pages in each pass during GIST VACUUM.

  7. Use full 64-bit XID for checking if a deleted GiST page is old enough.

  8. Allow VACUUM to be run with index cleanup disabled.

  9. Skip full index scan during cleanup of B-tree indexes when possible

  10. Do index FSM vacuuming sooner.

  11. Avoid early reuse of btree pages, causing incorrect query results.

  12. Clean up API for ambulkdelete/amvacuumcleanup as per today's discussion.