Horton

Re: type cache cleanup improvements

Alexander Korotkov <aekorotkov@gmail.com>

View thread
From: Alexander Korotkov <aekorotkov@gmail.com>
To: Pavel Borisov <pashkin.elfe@gmail.com>
Cc: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>, Andrei Lepikhov <lepihov@gmail.com>, jian he <jian.universality@gmail.com>, Artur Zakirov <zaartur@gmail.com>, Alexander Lakhin <exclusion@gmail.com>, Teodor Sigaev <teodor@sigaev.ru>, Pgsql Hackers <pgsql-hackers@postgresql.org>, Aleksander Alekseev <aleksander@timescale.com>, Tom Lane <tgl@sss.pgh.pa.us>, Michael Paquier <michael@paquier.xyz>
Date: 2024-10-22T17:33:24Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Maintain RelIdToTypeIdCacheHash in TypeCacheOpcCallback()

  2. Fix concurrrently in typcache_rel_type_cache.sql

  3. Avoid looping over all type cache entries in TypeCacheRelCallback()

  4. Update header comment for lookup_type_cache()

  5. Revert: Avoid looping over all type cache entries in TypeCacheRelCallback()

  6. Introduce hash_search_with_hash_value() function

  7. Optimize InvalidateAttoptCacheCallback() and TypeCacheTypCallback()

  8. Refactor initial hash lookup in dynahash.c

  9. Rationalize and improve error messages for some jsonpath items

  10. Avoid race in RelationBuildDesc() affecting CREATE INDEX CONCURRENTLY.

  11. Represent Lists as expansible arrays, not chains of cons-cells.

Attachments

On Tue, Oct 22, 2024 at 6:10 PM Pavel Borisov <pashkin.elfe@gmail.com> wrote:
> On Tue, 22 Oct 2024 at 11:34, Alexander Korotkov <aekorotkov@gmail.com> wrote:
>>
>> On Mon, Oct 21, 2024 at 2:30 PM Alexander Korotkov <aekorotkov@gmail.com> wrote:
>> >
>> > On Mon, Oct 21, 2024 at 1:16 PM Dagfinn Ilmari Mannsåker
>> > <ilmari@ilmari.org> wrote:
>> > > Alexander Korotkov <aekorotkov@gmail.com> writes:
>> > >
>> > > > On Mon, Oct 21, 2024 at 8:40 AM Andrei Lepikhov <lepihov@gmail.com> wrote:
>> > > >>
>> > > >> On 21/10/2024 06:32, Dagfinn Ilmari Mannsåker wrote:
>> > > >> > Alexander Korotkov <aekorotkov@gmail.com> writes:
>> > > >> >
>> > > >> >> +static Oid *in_progress_list;
>> > > >> >> +static int  in_progress_list_len;
>> > > >> >> +static int  in_progress_list_maxlen;
>> > > >> >
>> > > >> > Is there any particular reason not to use pg_list.h for this?
>> > > >> Sure. The type cache lookup has to be as much optimal as possible.
>> > > >> Using an array and relating sequential access to it, we avoid memory
>> > > >> allocations and deallocations 99.9% of the time. Also, quick access to
>> > > >> the single element (which we will have in real life almost all of the
>> > > >> time) is much faster than employing list machinery.
>> > >
>> > > Lists are actually dynamically resized arrays these days (see commit
>> > > 1cff1b95ab6ddae32faa3efe0d95a820dbfdc164), not linked lists, so
>> > > accessing arbitrary elements is O(1), not O(n). Just like this patch,
>> > > the size is doubled (starting at 16) whenever array is full.
>> > >
>> > > > +1,
>> > > > List with zero elements has to be NIL.  That means continuous
>> > > > allocations/deallocations.
>> > >
>> > > This however is a valid point (unless we keep a dummy zeroth element to
>> > > avoid it, which is even uglier than open-coding the array extension
>> > > logic), so objection withdrawn.
>> >
>> > OK, thank you!
>> >
>> > The attached revision fixes EXTRA_INSTALL in
>> > src/test/modules/typcache/Makefile.  Spotted off-list by Arthur
>> > Zakirov.
>>
>> I've re-checked that regression tests pass with
>> -DCLOBBER_CACHE_ALWAYS.  Also did some grammar corrections for
>> comments and commit message.  I'm going to push this if no objections.
>
> Thank you for working on this patch!
> Looked through the patchset once more.
>
> Patch 0001 (minor): "in the last" -> "after everything else" or "after other TypeCacheEntry contents"
>
> Patch 0002 looks ready to me.

Thank you, Pavel!  0001 revised according to your suggestion.

------
Regards,
Alexander Korotkov
Supabase