Horton

Re: type cache cleanup improvements

Alexander Korotkov <aekorotkov@gmail.com>

View thread
From: Alexander Korotkov <aekorotkov@gmail.com>
To: Andrei Lepikhov <lepihov@gmail.com>
Cc: Alexander Lakhin <exclusion@gmail.com>, Pavel Borisov <pashkin.elfe@gmail.com>, Teodor Sigaev <teodor@sigaev.ru>, Pgsql Hackers <pgsql-hackers@postgresql.org>, Aleksander Alekseev <aleksander@timescale.com>, Tom Lane <tgl@sss.pgh.pa.us>, Michael Paquier <michael@paquier.xyz>
Date: 2024-09-23T10:53:47Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Maintain RelIdToTypeIdCacheHash in TypeCacheOpcCallback()

  2. Fix concurrrently in typcache_rel_type_cache.sql

  3. Avoid looping over all type cache entries in TypeCacheRelCallback()

  4. Update header comment for lookup_type_cache()

  5. Revert: Avoid looping over all type cache entries in TypeCacheRelCallback()

  6. Introduce hash_search_with_hash_value() function

  7. Optimize InvalidateAttoptCacheCallback() and TypeCacheTypCallback()

  8. Refactor initial hash lookup in dynahash.c

  9. Rationalize and improve error messages for some jsonpath items

  10. Avoid race in RelationBuildDesc() affecting CREATE INDEX CONCURRENTLY.

  11. Represent Lists as expansible arrays, not chains of cons-cells.

On Wed, Sep 18, 2024 at 5:10 PM Andrei Lepikhov <lepihov@gmail.com> wrote:
> On 13/9/2024 01:38, Alexander Korotkov wrote:
> > I've tried to implement handling of concurrent invalidation similar to
> > commit fdd965d074.  However that appears to be more difficult that I
> > thought, because for some datatypes like arrays, ranges etc we might
> > need fill the element type and reference it.  So, I decided to
> > continue with the current approach but borrowing some ideas from
> > fdd965d074.  The revised patchset attached.
> Let me rephrase the issue in more straightforward terms to ensure we are
> all clear on the problem:
> The critical problem of the typcache lookup on not-yet-locked data is
> that it can lead to an inconsistent state of the TypEntry, potentially
> causing disruptions in the DBMS's operations, correct?
> Let's exemplify this statement. By filling typentry's lt_opr, eq_opr,
> and gt_opr fields, we access the AMOPSTRATEGY cache. One operation can
> successfully fetch data from the cache, but another can miss data and
> touch the catalogue table, causing invalidations. In this case, we can
> get an inconsistent set of operators. Do I understand the problem
> statement correctly?

Actually, I didn't research much if there is a material problem.  So,
I didn't try to concurrently delete some operator class members
concurrently to lookup_type_cache().  There are probably some bugs,
but they likely have low impact in practice, given that type/opclass
changes are very rare.

Yet I was concentrated on why do lookup_type_cache() returns
TypeCacheEntry filled with whatever caller asked given there could be
concurrent invalidations.

So, my approach was to
1) Document how we currently handle concurrent invalidations.
2) Maintain RelIdToTypeIdCacheHash correctly with concurrent invalidations.

------
Regards,
Alexander Korotkov
Supabase