Horton

Re: [PoC] Improve dead tuple storage for lazy vacuum

John Naylor <john.naylor@enterprisedb.com>

View thread View on postgr.es
From: John Naylor <john.naylor@enterprisedb.com>
To: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Nathan Bossart <nathandbossart@gmail.com>, Andres Freund <andres@anarazel.de>, Matthias van de Meent <boekewurm+postgres@gmail.com>, Yura Sokolov <y.sokolov@postgrespro.ru>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2023-01-09T08:59:04Z
Lists: pgsql-hackers

Attachments

> [working on templating]

In the end, I decided to base my effort on v8, and not v12 (based on one of
my less-well-thought-out ideas). The latter was a good experiment, but it
did not lead to an increase in readability as I had hoped. The attached v17
is still rough, but it's in good enough shape to evaluate a mostly-complete
templating implementation.

Part of what I didn't like about v8 was distinctions like "node" vs
"nodep", which hinder readability. I've used "allocnode" for some cases
where it makes sense, which is translated to "newnode" for the local
pointer. Some places I just gave up and used "nodep" for parameters like in
v8, just to get it done. We can revisit naming later.

Not done yet:

- get_handle() is not implemented
- rt_attach is defined but unused
- grow_node_kind() was hackishly removed, but could be turned into a macro
(or function that writes to 2 pointers)
- node_update_inner() is back, now that we can share a template with
"search". Seems easier to read, and I suspect this is easier for the
compiler.
- the value type should really be a template macro, but is still hard-coded
to uint64
- I think it's okay if the key is hard coded for PG16: If some use case
needs more than uint64, we could consider "single-value leaves" with varlen
keys as a template option.
- benchmark tests not updated

v13-0007 had some changes to the regression tests, but I haven't included
those. The tests from v13-0003 do pass, both locally and shared. I quickly
hacked together changing shared/local tests by hand (need to recompile),
but it would be good for maintainability if tests could run once each with
local and shmem, but use the same "expected" test output.

Also, I didn't look to see if there were any changes in v14/15 that didn't
have to do with precise memory accounting.

At this point, Masahiko, I'd appreciate your feedback on whether this is an
improvement at all (or at least a good base for improvement), especially
for integrating with the TID store. I think there are some advantages to
the template approach. One possible disadvantage is needing separate
functions for each local and shared memory.

If we go this route, I do think the TID store should invoke the template as
static functions. I'm not quite comfortable with a global function that may
not fit well with future use cases.

One review point I'll mention: Somehow I didn't notice there is no use for
the "chunk" field in the rt_node type -- it's only set to zero and copied
when growing. What is the purpose? Removing it would allow the
smallest node to take up only 32 bytes with a fanout of 3, by eliminating
padding.

Also, v17-0005 has an optimization/simplification for growing into node125
(my version needs an assertion or fallback, but works well now), found by
another reading of Andres' prototype There is a lot of good engineering
there, we should try to preserve it.

--
John Naylor
EDB: http://www.enterprisedb.com

Commits

  1. radixtree: Fix SIGSEGV at update of embeddable value to non-embeddable.

  2. Get rid of anonymous struct

  3. Teach radix tree to embed values at runtime

  4. Teach TID store to skip bitmap for small numbers of offsets

  5. Use bump context for TID bitmaps stored by vacuum

  6. Fix alignment of stack variable

  7. Use TidStore for dead tuple TIDs storage during lazy vacuum.

  8. Rethink create and attach APIs of shared TidStore.

  9. Fix inconsistent function prototypes with function definitions.

  10. Fix a calculation in TidStoreCreate().

  11. Fix potential integer handling issue in radixtree.h.

  12. Add TIDStore, to store sets of TIDs (ItemPointerData) efficiently.

  13. Fix link error for test_radixtree module on Windows

  14. Blind attempt to fix ODR violations

  15. Fix incorrect format specifier for int64

  16. Fix redefinition of typedefs

  17. Add template for adaptive radix tree

  18. Fix signedness error in 9f225e992 for gcc

  19. Introduce helper SIMD functions for small byte arrays

  20. Optimize vacuuming of relations with no indexes.

  21. Add bound check before bsearch() for performance

  22. Allocate consecutive blocks during parallel seqscans