Horton
Menu

Re: Better shared data structure management and resizable shared data structures

Robert Haas <robertmhaas@gmail.com>

View thread
From: Robert Haas <robertmhaas@gmail.com>
To: Heikki Linnakangas <hlinnaka@iki.fi>
Cc: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, Andres Freund <andres@anarazel.de>, pgsql-hackers <pgsql-hackers@postgresql.org>, chaturvedipalak1911@gmail.com
Date: 2026-03-12T18:56:18Z
Lists: pgsql-hackers 
On Thu, Mar 12, 2026 at 2:41 PM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
> shmem_startup_hook() is too late. The shmem structs need to be
> registered at postmaster startup before the shmem segment is allocated,
> so that we can calculate the total size needed.

Sorry, I meant shmem_request_hook.

> I'm currently leaning towards _PG_init(), except for allocations that
> depend on MaxBackends. For those, you can install a shmem_request_hook
> that sets the size in the descriptor. In other words, you can leave the
> 'size' as empty in _PG_init(), but set it later in the shmem_request_hook.

Why can't you just do the whole thing later?

> Another option is to add a new bespoken callback in the descriptor for
> such size adjustments, which would get called at the same time as
> shmem_request_hook. That might be a little more ergonomic, there would
> no longer be any need for extensions to use the old
> shmem_request/startup_hooks with the new ShmemRegisterStruct() mechanism.

Yeah, worth considering.

> Except that you'd still need them for RequestNamedLWLockTranche(). I
> wonder if we should recommend extensions to embed the LWLock struct into
> their shared memory struct and use the LWLockInitialize() and
> LWLockNewTrancheId() functions instead. That fits the new
> ShmemRegisterStruct() API a little better than RequestNamedLWLockTranche().

Yeah, I think RequestNamedLWLockTranche() might be fine if you just
need LWLocks, but if you need a bunch of resources, putting them all
into the same chunk of memory seems cleaner.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Tidy up #ifdef USE_INJECTION_POINTS guards

  2. Convert all remaining subsystems to use the new shmem allocation API

  3. Convert buffer manager to use the new shmem allocation functions

  4. Add alignment option to ShmemRequestStruct()

  5. Convert AIO to use the new shmem allocation functions

  6. Convert SLRUs to use the new shmem allocation functions

  7. Refactor shmem initialization code in predicate.c

  8. Use the new shmem allocation functions in a few core subsystems

  9. Convert lwlock.c to use the new shmem allocation functions

  10. Introduce a registry of built-in shmem subsystems

  11. Convert pg_stat_statements to use the new shmem allocation functions

  12. Add a test module to test after-startup shmem allocations

  13. Introduce a new mechanism for registering shared memory areas

  14. Move some code from shmem.c and shmem.h

  15. Improve test_lwlock_tranches

  16. Test pg_stat_statements across crash restart

  17. Refactor PredicateLockShmemInit to not reuse var for different things

  18. Refactor ShmemIndex initialization

  19. Add a new shmem_request_hook hook.