Horton
Menu

Re: Better shared data structure management and resizable shared data structures

Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>

View thread
From: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
To: Heikki Linnakangas <hlinnaka@iki.fi>
Cc: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>, Robert Haas <robertmhaas@gmail.com>, Andres Freund <andres@anarazel.de>, pgsql-hackers <pgsql-hackers@postgresql.org>, chaturvedipalak1911@gmail.com
Date: 2026-04-07T14:19:28Z
Lists: pgsql-hackers

Attachments

Hi Heikki,
CallShmemCallbacksAfterStartup() holds ShmemIndexLock while invoking
init_fn/attach_fn callbacks. That looks wrong. Before this commit,
init or attach code was not run with the lock held. Any reason the
lock is held while calling init and attach callbacks. Since these
function can come from extensions, we don't have control on what goes
in those functions, and thus looks problematic. Further, it will
serialize all the attach_fn executions across backends, since each
will be run under the lock. In my case, the init_fn was performing
ShmemIndex lookup which deadlocked. It's questionable whether init
function should lookup ShmemIndex but, it's not something that needs
to be prohibited either.

Here's patch fixing it.

--
Best Wishes,
Ashutosh Bapat

On Tue, Apr 7, 2026 at 6:56 PM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
>
> On 07/04/2026 15:24, Dagfinn Ilmari Mannsåker wrote:
> > Heikki Linnakangas <hlinnaka@iki.fi> writes:
> >
> >> Those are now committed, and here's a new version rebased over those
> >> changes.
> >
> > I noticed this bit during my habitual morning skim of new commits:
> >
> >> diff --git a/src/backend/utils/misc/injection_point.c b/src/backend/utils/misc/injection_point.c
> >> index c06b0e9b800..9981d6e212f 100644
> >> --- a/src/backend/utils/misc/injection_point.c
> >> +++ b/src/backend/utils/misc/injection_point.c
> >> @@ -17,6 +17,7 @@
> >>    */
> >>   #include "postgres.h"
> >>
> >> +#include "storage/subsystems.h"
> >>   #include "utils/injection_point.h"
> >>
> >>   #ifdef USE_INJECTION_POINTS
> >> @@ -109,6 +110,11 @@ typedef struct InjectionPointCacheEntry
> >>
> >>   static HTAB *InjectionPointCache = NULL;
> >>
> >> +#ifdef USE_INJECTION_POINTS
> >> +static void InjectionPointShmemRequest(void *arg);
> >> +static void InjectionPointShmemInit(void *arg);
> >> +#endif
> >> +
> >
> > This is already inside an `#ifdef USE_INJECTION_POINTS` guard (in fact
> > visible at the end of the previous diff hunk), no need for another one.
>
> Fixed, thanks. I also noticed that the #include "storage/subsystems.h"
> can be moved inside the #ifdef block; fixed that too.
>
> - Heikki
>

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Tidy up #ifdef USE_INJECTION_POINTS guards

  2. Convert all remaining subsystems to use the new shmem allocation API

  3. Convert buffer manager to use the new shmem allocation functions

  4. Add alignment option to ShmemRequestStruct()

  5. Convert AIO to use the new shmem allocation functions

  6. Convert SLRUs to use the new shmem allocation functions

  7. Refactor shmem initialization code in predicate.c

  8. Use the new shmem allocation functions in a few core subsystems

  9. Convert lwlock.c to use the new shmem allocation functions

  10. Introduce a registry of built-in shmem subsystems

  11. Convert pg_stat_statements to use the new shmem allocation functions

  12. Add a test module to test after-startup shmem allocations

  13. Introduce a new mechanism for registering shared memory areas

  14. Move some code from shmem.c and shmem.h

  15. Improve test_lwlock_tranches

  16. Test pg_stat_statements across crash restart

  17. Refactor PredicateLockShmemInit to not reuse var for different things

  18. Refactor ShmemIndex initialization

  19. Add a new shmem_request_hook hook.