Horton

RE: Changing shared_buffers without restart

Jack Ng <jack.ng@huawei.com>

View thread
From: Jack Ng <Jack.Ng@huawei.com>
To: Dmitry Dolgov <9erthalion6@gmail.com>
Cc: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>, Robert Haas <robertmhaas@gmail.com>, Ni Ku <jakkuniku@gmail.com>
Date: 2025-05-07T05:34:37Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Remove PG_MMAP_FLAGS from mem.h

  2. Improve runtime and output of tests for replication slots checkpointing.

  3. Revert support for improved tracking of nested queries

  4. Use exported symbols list on macOS for loadable modules as well

  5. Add support for basic NUMA awareness

  6. Avoid unnecessary copying of a string in pg_restore.c

  7. aio: Infrastructure for io_method=worker

  8. Improve InitShmemAccess() prototype

> all the possible scenarios. But now I'm reworking it along the lines suggested
> by Thomas, and will address those as well. Thanks!

Thanks for the info, Dmitry.
Just want to confirm my understanding of Thomas' suggestion and your discussions... I think the simpler and more portable solution goes something like the following? 

* For each BP resource segment (main, desc, buffers, etc):
    1. create an anonymous file as backing
    2. mmap a large reserved shared memory area with PROTO_READ/WRITE + MAP_NORESERVE using the anon fd
    3. use ftruncate to back the in-use region (and maybe posix_fallocate too to avoid SIGBUS on alloc failure during first-touch), but no need to create a memory mapping for it
    4. also no need to create a separate mapping for the reserved region (already covered by the mapping created in 2.)

|-- Memory mapping (MAP_NORESERVE) for BUFFER --|
|-- In-use region --|----- Reserved region -----|

* During resize, simply calculate the new size and call ftruncate on each segment to adjust memory accordingly, no need to mmap/munmap or modify any memory mapping.

I tried this approach with a test program (with huge pages), and both expand and shrink seem to work as expected --for shrink, the memory is freed right after the resize ftruncate.

Regards,

Jack Ng