Horton

Re: type cache cleanup improvements

Alexander Korotkov <aekorotkov@gmail.com>

View thread
From: Alexander Korotkov <aekorotkov@gmail.com>
To: jian he <jian.universality@gmail.com>
Cc: Artur Zakirov <zaartur@gmail.com>, Andrei Lepikhov <lepihov@gmail.com>, Alexander Lakhin <exclusion@gmail.com>, Pavel Borisov <pashkin.elfe@gmail.com>, Teodor Sigaev <teodor@sigaev.ru>, Pgsql Hackers <pgsql-hackers@postgresql.org>, Aleksander Alekseev <aleksander@timescale.com>, Tom Lane <tgl@sss.pgh.pa.us>, Michael Paquier <michael@paquier.xyz>
Date: 2024-10-20T22:09:23Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Maintain RelIdToTypeIdCacheHash in TypeCacheOpcCallback()

  2. Fix concurrrently in typcache_rel_type_cache.sql

  3. Avoid looping over all type cache entries in TypeCacheRelCallback()

  4. Update header comment for lookup_type_cache()

  5. Revert: Avoid looping over all type cache entries in TypeCacheRelCallback()

  6. Introduce hash_search_with_hash_value() function

  7. Optimize InvalidateAttoptCacheCallback() and TypeCacheTypCallback()

  8. Refactor initial hash lookup in dynahash.c

  9. Rationalize and improve error messages for some jsonpath items

  10. Avoid race in RelationBuildDesc() affecting CREATE INDEX CONCURRENTLY.

  11. Represent Lists as expansible arrays, not chains of cons-cells.

Attachments

On Sun, Oct 20, 2024 at 9:00 PM Alexander Korotkov <aekorotkov@gmail.com> wrote:
> On Sun, Oct 20, 2024 at 8:47 PM Alexander Korotkov <aekorotkov@gmail.com> wrote:
> > On Tue, Oct 15, 2024 at 12:50 PM jian he <jian.universality@gmail.com> wrote:
> > > build from source, DCLOBBER_CACHE_ALWAYS takes a very long time.
> > > So I gave up.
> > >
> > >
> > > in lookup_type_cache, we unconditionally do
> > > in_progress_list_len++;
> > > in_progress_list_len--;
> >
> > Yes, this should work OK when no errors.  On error or interruption,
> > finalize_in_progress_typentries() will clean the things up.
> >
> > > "static int    in_progress_list_len;"
> > > means in_progress_list_len value change is confined in
> > > src/backend/utils/cache/typcache.c.
> >
> > Yep.
> >
> > > based on above information, i am still  confused with
> > > cleanup_in_progress_typentries, in_progress_list_len
> > > is there any simple sql example to demo
> > > cleanup_in_progress_typentries,  in_progress_list_len> 0.
> >
> > I don't think there is simple sql to reliably reproduce that.  In
> > order to hit that, we must process invalidation messages in some
> > (short) moment of time during lookup_type_cache().  You can reproduce
> > that by setting a breakpoint in lookup_type_cache() and in parallel do
> > something to invalidate the type cache entry (for instance, ALTER
> > TABLE ... ADD COLUMN ... would invalidate the composite type).
>
> Oops, concurrent invalidation message is not enough here.  So,
> -DCLOBBER_CACHE_ALWAYS is also not enough to reproduce the situation.
> Injection-point test is required.  I'm going to add this.

Here you go.  The test with injection point is implemented.

------
Regards,
Alexander Korotkov
Supabase