Horton

Re: [PoC] Improve dead tuple storage for lazy vacuum

John Naylor <john.naylor@enterprisedb.com>

View thread View on postgr.es
From: John Naylor <john.naylor@enterprisedb.com>
To: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Nathan Bossart <nathandbossart@gmail.com>, Andres Freund <andres@anarazel.de>, Matthias van de Meent <boekewurm+postgres@gmail.com>, Yura Sokolov <y.sokolov@postgrespro.ru>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2023-03-10T06:42:33Z
Lists: pgsql-hackers 
On Thu, Mar 9, 2023 at 1:51 PM Masahiko Sawada <sawada.mshk@gmail.com>
wrote:

> I've attached the new version patches. I merged improvements and fixes
> I did in the v29 patch.

I haven't yet had a chance to look at those closely, since I've had to
devote time to other commitments. I remember I wasn't particularly
impressed that v29-0008 mixed my requested name-casing changes with a bunch
of other random things. Separating those out would be an obvious way to
make it easier for me to look at, whenever I can get back to this. I need
to look at the iteration changes as well, in addition to testing memory
measurement (thanks for the new results, they look encouraging).

> Apart from the memory measurement stuff, I've done another todo item
> on my list; adding min max classes for node3 and node125. I've done

This didn't help us move us closer to something committable the first time
you coded this without making sure it was a good idea. It's still not
helping and arguably makes it worse. To be fair, I did speak positively
about _considering_ additional size classes some months ago, but that has a
very obvious maintenance cost, something we can least afford right now.

I'm frankly baffled you thought this was important enough to work on again,
yet thought it was a waste of time to try to prove to ourselves that
autovacuum in a realistic, non-deterministic workload gave the same answer
as the current tid lookup. Even if we had gone that far, it doesn't seem
like a good idea to add non-essential code to critical paths right now.

We're rapidly running out of time, and we're at the point in the cycle
where it's impossible to get meaningful review from anyone not already
intimately familiar with the patch series. I only want to see progress on
addressing possible (especially architectural) objections from the
community, because if they don't notice them now, they surely will after
commit. I have my own list of possible objections as well as bikeshedding
points, which I'll clean up and share next week. I plan to invite Andres to
look at that list and give his impressions, because it's a lot quicker than
reading the patches. Based on that, I'll hopefully be able to decide
whether we have enough time to address any feedback and do remaining
polishing in time for feature freeze.

I'd suggest sharing your todo list in the meanwhile, it'd be good to
discuss what's worth doing and what is not.

--
John Naylor
EDB: http://www.enterprisedb.com

Commits

  1. radixtree: Fix SIGSEGV at update of embeddable value to non-embeddable.

  2. Get rid of anonymous struct

  3. Teach radix tree to embed values at runtime

  4. Teach TID store to skip bitmap for small numbers of offsets

  5. Use bump context for TID bitmaps stored by vacuum

  6. Fix alignment of stack variable

  7. Use TidStore for dead tuple TIDs storage during lazy vacuum.

  8. Rethink create and attach APIs of shared TidStore.

  9. Fix inconsistent function prototypes with function definitions.

  10. Fix a calculation in TidStoreCreate().

  11. Fix potential integer handling issue in radixtree.h.

  12. Add TIDStore, to store sets of TIDs (ItemPointerData) efficiently.

  13. Fix link error for test_radixtree module on Windows

  14. Blind attempt to fix ODR violations

  15. Fix incorrect format specifier for int64

  16. Fix redefinition of typedefs

  17. Add template for adaptive radix tree

  18. Fix signedness error in 9f225e992 for gcc

  19. Introduce helper SIMD functions for small byte arrays

  20. Optimize vacuuming of relations with no indexes.

  21. Add bound check before bsearch() for performance

  22. Allocate consecutive blocks during parallel seqscans