Horton
Menu

Re: Better shared data structure management and resizable shared data structures

Robert Haas <robertmhaas@gmail.com>

View thread
From: Robert Haas <robertmhaas@gmail.com>
To: Heikki Linnakangas <hlinnaka@iki.fi>
Cc: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, Andres Freund <andres@anarazel.de>, pgsql-hackers <pgsql-hackers@postgresql.org>, chaturvedipalak1911@gmail.com
Date: 2026-03-12T17:03:21Z
Lists: pgsql-hackers 
On Fri, Mar 6, 2026 at 9:13 AM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
> 1. _PG_init() gets called early at postmaster startup, if the library is
> in shared_preload_libraries. If it's not in shared_preload_libraries, it
> gets called whenever the module is loaded.
>
> 2. The library can install a shmem_request_hook, which gets called early
> at postmaster startup, but after initializing the MaxBackends GUC. It
> only gets called when the library is loaded via shared_preload_libraries.
>
> 3. The library can install a shmem_startup_hook. It gets called later at
> postmaster startup, after the shared memory segment has been allocated.
> In EXEC_BACKEND mode it also gets called at backend startup. It does not
> get called if the library is not listed in shared_preload_libraries.
>
> None of these is quite the right moment to call the new
> ShmemRegisterStruct() function. _PG_init() is too early if the extension
> needs MaxBackends for sizing the shared memory area. shmem_request_hook
> is otherwise good, but in EXEC_BACKEND mode, the ShmemRegisterStruct()
> function needs to also be called backend startup and shmem_request_hook
> is not called at backend startup. shmem_startup_hook() is too late.

I believe that the design goal of
4f2400cb3f10aa79f99fba680c198237da28dd38 was to make it so that people
who had working extensions already didn't need to change their code,
but those for whom the restrictions of doing things in _PG_init were
annoying would have a workable alternative. I think that's a pretty
good goal, although I don't feel we absolutely have to stick to it. It
could easily be worth breaking that if we get something cool out of
it. But is there a reason we can't make it so that this new mechanism
can be used either from _PG_init() or shmem_startup_hook()? (I assume
there is or you likely would have done it already, but it's not clear
to me what that reason is.)

-- 
Robert Haas
EDB: http://www.enterprisedb.com

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Tidy up #ifdef USE_INJECTION_POINTS guards

  2. Convert all remaining subsystems to use the new shmem allocation API

  3. Convert buffer manager to use the new shmem allocation functions

  4. Add alignment option to ShmemRequestStruct()

  5. Convert AIO to use the new shmem allocation functions

  6. Convert SLRUs to use the new shmem allocation functions

  7. Refactor shmem initialization code in predicate.c

  8. Use the new shmem allocation functions in a few core subsystems

  9. Convert lwlock.c to use the new shmem allocation functions

  10. Introduce a registry of built-in shmem subsystems

  11. Convert pg_stat_statements to use the new shmem allocation functions

  12. Add a test module to test after-startup shmem allocations

  13. Introduce a new mechanism for registering shared memory areas

  14. Move some code from shmem.c and shmem.h

  15. Improve test_lwlock_tranches

  16. Test pg_stat_statements across crash restart

  17. Refactor PredicateLockShmemInit to not reuse var for different things

  18. Refactor ShmemIndex initialization

  19. Add a new shmem_request_hook hook.