Horton

Re: Changing shared_buffers without restart

Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>

View thread
From: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
To: Dmitry Dolgov <9erthalion6@gmail.com>
Cc: Thomas Munro <thomas.munro@gmail.com>, pgsql-hackers@postgresql.org, Robert Haas <robertmhaas@gmail.com>
Date: 2025-09-29T06:51:08Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Remove PG_MMAP_FLAGS from mem.h

  2. Improve runtime and output of tests for replication slots checkpointing.

  3. Revert support for improved tracking of nested queries

  4. Use exported symbols list on macOS for loadable modules as well

  5. Add support for basic NUMA awareness

  6. Avoid unnecessary copying of a string in pg_restore.c

  7. aio: Infrastructure for io_method=worker

  8. Improve InitShmemAccess() prototype

On Sun, Sep 28, 2025 at 2:54 PM Dmitry Dolgov <9erthalion6@gmail.com> wrote:
>
> > On Thu, Sep 18, 2025 at 10:25:29AM +0530, Ashutosh Bapat wrote:
> > Given these things, I think we should set up the buffer lookup table
> > to hold maximum entries required to expand the buffer pool to its
> > maximum, right at the beginning.
>
> Thanks for investigating. I think another option would be to rebuild the
> buffer lookup table (create a new table based on the new size and copy
> the data over from the original one) as part of the resize procedure,
> alongsize with buffers eviction and initialization. From what I recall
> the size of buffer lookup table is about two orders of magnitude lower
> than shared buffers, so the overhead should not be that large even for
> significant amount of buffers.

The proposal will work but will require significant work:

1. The pointer to the shared buffer lookup table will change. The
change needs to be absorbed by all the processes at the same time; we
can not have few processes accessing old lookup table and few
processes new one. That has potential to make many processes wait for
a very long time. That can be fixed by accessing a new pointer when
the next buffer lookup access happens by modifying BufTable*
functions. But that means an extra condition checks and some extra
code in those hot paths. Not sure whether that's acceptable.
2. The memory consumed by the old buffer lookup table will need to be
"freed" to the OS. The only way to do so is by having a new memory
segment (which can be unmapped) or unmapping portions of segment
dedicated to the buffer lookup table. That's some more synchronization
and additional wait times for backends.
3. When the new shared buffer lookup table will be built, processes
may be able to access it in shared mode but they may not be able to
make changes to it (or else we need to make corresponding changes to
new table as well). That means more restrictions on the running
backends.

I am not saying that we can not implement your idea, but maybe we
could do that incrementally after basic resizing is in place.

--
Best Wishes,
Ashutosh Bapat