Horton
Menu

Re: Better shared data structure management and resizable shared data structures

Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>

View thread
From: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
To: Heikki Linnakangas <hlinnaka@iki.fi>
Cc: Matthias van de Meent <boekewurm+postgres@gmail.com>, Robert Haas <robertmhaas@gmail.com>, Andres Freund <andres@anarazel.de>, pgsql-hackers <pgsql-hackers@postgresql.org>, chaturvedipalak1911@gmail.com
Date: 2026-04-07T10:06:25Z
Lists: pgsql-hackers

Attachments

On Mon, Apr 6, 2026 at 7:23 PM Ashutosh Bapat
<ashutosh.bapat.oss@gmail.com> wrote:
>
> I have kept these two patches separate from the main patch so that I
> can remove them if others feel they are not worth including in the
> feature.

Here are patches rebased on the latest HEAD. No conflicts just rebase.

Here are differences from the previous patchset.

o. There are two patches in this patchset now. a. 0001 which supports
resizable shared memory and is equivalent to 0001 + 0002 + 0004 + 0005
from the previous patchset. b. 0002 which is 0006 from the previous
patchset and adds support for protecting resizable shared memory
structures. 0003, which added diagnostics to investigate CFBot
failure, from the previous patchset is not required anymore since all
tests pass with CFBot.

o. I have merged 0002 into 0001 from the previous patchset since with
that patch all platforms are green on CFBot. The resizable shared
memory test now uses /proc/self/smaps instead of /proc/self/status to
find the amount of memory allocated in the main shared memory segment
of PostgreSQL.

o. Merged 0004, which supported minimum_size, into 0001. Minimum_size
would be useful to protect against accidental shrinkage of the
resizable structures. It will help additional support for minimum
sizes of GUCs like shared_buffers. It also makes it easy and intuitive
to distinguish between fixed-size and resizable structures, and will
be useful to find the minimum size of the shared memory segment.

o. Merged 0005, which allows ABI compatibility between the binaries
which support resizable shared memory and those which don't, into
0001. Apart from ABI compatibility, the code has lesser #ifdef blocks
and thus easier to read and maintain.

I didn't find it useful to keep 0004 and 0005 separate since they were
interdependent and made review complicated and have higher chances of
being acceptable.

o. 0006 is still separate since I am not sure whether the
functionality is absolutely needed at this time. In an offlist
discussion, Andres mentioned that it is not strictly needed. The
subsystem that uses the resizable shared memory can implement their
own protection if required and integrate it in the subsystems specific
synchronization. But Matthias thinks different. The API to add
protection is platform dependent, so it's better to abstract it via
shmem.c. If we decide to accept this patch, we should merge it into
0001 before committing.

Also did some more cleanups and changed the name of the GUC
have_resizable_shmem to have_resizable_shared_memory since shmem is an
internal phrase.

I am looking at merging the resizable_shmem module into test_shmem module next.

-- 
Best Wishes,
Ashutosh Bapat

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Tidy up #ifdef USE_INJECTION_POINTS guards

  2. Convert all remaining subsystems to use the new shmem allocation API

  3. Convert buffer manager to use the new shmem allocation functions

  4. Add alignment option to ShmemRequestStruct()

  5. Convert AIO to use the new shmem allocation functions

  6. Convert SLRUs to use the new shmem allocation functions

  7. Refactor shmem initialization code in predicate.c

  8. Use the new shmem allocation functions in a few core subsystems

  9. Convert lwlock.c to use the new shmem allocation functions

  10. Introduce a registry of built-in shmem subsystems

  11. Convert pg_stat_statements to use the new shmem allocation functions

  12. Add a test module to test after-startup shmem allocations

  13. Introduce a new mechanism for registering shared memory areas

  14. Move some code from shmem.c and shmem.h

  15. Improve test_lwlock_tranches

  16. Test pg_stat_statements across crash restart

  17. Refactor PredicateLockShmemInit to not reuse var for different things

  18. Refactor ShmemIndex initialization

  19. Add a new shmem_request_hook hook.