Horton
Menu

Re: Better shared data structure management and resizable shared data structures

Heikki Linnakangas <hlinnaka@iki.fi>

View thread
From: Heikki Linnakangas <hlinnaka@iki.fi>
To: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Cc: Robert Haas <robertmhaas@gmail.com>, Andres Freund <andres@anarazel.de>, pgsql-hackers <pgsql-hackers@postgresql.org>, chaturvedipalak1911@gmail.com
Date: 2026-03-30T20:15:44Z
Lists: pgsql-hackers 
On 30/03/2026 07:50, Ashutosh Bapat wrote:
> On Sat, Mar 28, 2026 at 4:47 AM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
>> On 27/03/2026 09:01, Ashutosh Bapat wrote:
>>> /* Restore basic shared memory pointers */
>>> if (UsedShmemSegAddr != NULL)
>>> + {
>>> InitShmemAllocator(UsedShmemSegAddr);
>>> + ShmemCallRequestCallbacks();
>>>
>>> It's not clear how we keep the list of registered callbacks across the
>>> backends and also after restart in-sync. How do we make sure that the
>>> callbacks registered at this time are the same callbacks registered
>>> before creating the shared memory? How do we make sure that the
>>> callbacks registered after the startup are also registered after
>>> restart?
>>
>> On Unix systems, the registered callbacks are inherited by fork(), and
>> also survive over crash restart. With EXEC_BACKEND, the assumption is
>> that calling a library's _PG_init() function will register the same
>> callbacks every time. We make the same assumption today with the
>> shmem_startup hook.
> 
> RegisterShmemCallbacks() may be called after the startup, and it will
> add new areas to the shared memory. How are those registries synced
> across the backends? From your answer below, those registries are not
> synced across backends. They will be wiped out by the restart and
> won't be registered again. Is that right? I think we need to document
> this fact and also the need to call RegisterShmemCallbacks() from all
> the backends where the new areas are required after the startup.

Correct. Ok, I'll add a note to comment on RegisterShmemCallbacks() to 
call that out more explicitly, hope it helps.

- Heikki

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Tidy up #ifdef USE_INJECTION_POINTS guards

  2. Convert all remaining subsystems to use the new shmem allocation API

  3. Convert buffer manager to use the new shmem allocation functions

  4. Add alignment option to ShmemRequestStruct()

  5. Convert AIO to use the new shmem allocation functions

  6. Convert SLRUs to use the new shmem allocation functions

  7. Refactor shmem initialization code in predicate.c

  8. Use the new shmem allocation functions in a few core subsystems

  9. Convert lwlock.c to use the new shmem allocation functions

  10. Introduce a registry of built-in shmem subsystems

  11. Convert pg_stat_statements to use the new shmem allocation functions

  12. Add a test module to test after-startup shmem allocations

  13. Introduce a new mechanism for registering shared memory areas

  14. Move some code from shmem.c and shmem.h

  15. Improve test_lwlock_tranches

  16. Test pg_stat_statements across crash restart

  17. Refactor PredicateLockShmemInit to not reuse var for different things

  18. Refactor ShmemIndex initialization

  19. Add a new shmem_request_hook hook.