Horton
Menu

Re: Better shared data structure management and resizable shared data structures

Heikki Linnakangas <hlinnaka@iki.fi>

View thread
From: Heikki Linnakangas <hlinnaka@iki.fi>
To: Matthias van de Meent <boekewurm+postgres@gmail.com>
Cc: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, Robert Haas <robertmhaas@gmail.com>, Andres Freund <andres@anarazel.de>, pgsql-hackers <pgsql-hackers@postgresql.org>, chaturvedipalak1911@gmail.com
Date: 2026-04-05T19:35:58Z
Lists: pgsql-hackers 
On 04/04/2026 16:51, Matthias van de Meent wrote:
>> +++ b/src/include/storage/shmem.h
>> +/*
>> + * Shared memory is reserved and allocated in stages at postmaster startup,
>> + * and in EXEC_BACKEND mode, there's some extra work done to "attach" to them
>> + * at backend startup.  ShmemCallbacks holds callback functions that are
>> + * called at different stages.
>> + */
>> +typedef struct ShmemCallbacks
> 
> Maybe this should also have the opportunity for a (before_)shmem_exit callback?

Hmm, yeah, perhaps, but I'm going to skip that for now. We already have 
a mechanism for shmem-exit callbacks, and I'm not sure how that would 
plug into this. I think we can do that later if it turns out to be a 
good idea, and I don't think it changes the parts that's included in the 
patches now.

>> + * on-demaind in a backend.  If a subsystem sets this flag, the callbacks are
>> + * called immediately after registration, to initialize or attach to the
>> + * requested shared memory areas.
> 
> Ideally we only immediately call the callbacks if we're under
> postmaster, or in a standalone backend; we shouldn't allocate shmem
> for some preloaded libraries that set this flag, at least not ahead of
> loading all preload libraries.

Right, the SHMEM_CALLBACKS_ALLOW_AFTER_STARTUP flag doesn't do anything 
if called during shared_preload_libraries processing. I'll re-word the 
comment to clarify that.

> While it's mostly mechanical changes, it did make me notice the rather
> annoying allocation patterns by XLOGShmemRequest. It allocates various
> types of data in one go (which, in principle, is fine) but in doing so
> it adds its own alignment tricks etc, and I'm not super stoked about
> that. If time allows, could we clean that up?

I'm not going to cram it into these patches, but +1.

- Heikki

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Tidy up #ifdef USE_INJECTION_POINTS guards

  2. Convert all remaining subsystems to use the new shmem allocation API

  3. Convert buffer manager to use the new shmem allocation functions

  4. Add alignment option to ShmemRequestStruct()

  5. Convert AIO to use the new shmem allocation functions

  6. Convert SLRUs to use the new shmem allocation functions

  7. Refactor shmem initialization code in predicate.c

  8. Use the new shmem allocation functions in a few core subsystems

  9. Convert lwlock.c to use the new shmem allocation functions

  10. Introduce a registry of built-in shmem subsystems

  11. Convert pg_stat_statements to use the new shmem allocation functions

  12. Add a test module to test after-startup shmem allocations

  13. Introduce a new mechanism for registering shared memory areas

  14. Move some code from shmem.c and shmem.h

  15. Improve test_lwlock_tranches

  16. Test pg_stat_statements across crash restart

  17. Refactor PredicateLockShmemInit to not reuse var for different things

  18. Refactor ShmemIndex initialization

  19. Add a new shmem_request_hook hook.