Horton

Re: Changing shared_buffers without restart

Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>

View thread
From: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
To: Dmitry Dolgov <9erthalion6@gmail.com>
Cc: Thomas Munro <thomas.munro@gmail.com>, pgsql-hackers@postgresql.org, Robert Haas <robertmhaas@gmail.com>
Date: 2025-10-01T10:20:17Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Remove PG_MMAP_FLAGS from mem.h

  2. Improve runtime and output of tests for replication slots checkpointing.

  3. Revert support for improved tracking of nested queries

  4. Use exported symbols list on macOS for loadable modules as well

  5. Add support for basic NUMA awareness

  6. Avoid unnecessary copying of a string in pg_restore.c

  7. aio: Infrastructure for io_method=worker

  8. Improve InitShmemAccess() prototype

On Wed, Oct 1, 2025 at 2:40 PM Dmitry Dolgov <9erthalion6@gmail.com> wrote:
>
> > On Mon, Sep 29, 2025 at 12:21:08PM +0530, Ashutosh Bapat wrote:
> > On Sun, Sep 28, 2025 at 2:54 PM Dmitry Dolgov <9erthalion6@gmail.com> wrote:
> > >
> > > > On Thu, Sep 18, 2025 at 10:25:29AM +0530, Ashutosh Bapat wrote:
> > > > Given these things, I think we should set up the buffer lookup table
> > > > to hold maximum entries required to expand the buffer pool to its
> > > > maximum, right at the beginning.
> > >
> > > Thanks for investigating. I think another option would be to rebuild the
> > > buffer lookup table (create a new table based on the new size and copy
> > > the data over from the original one) as part of the resize procedure,
> > > alongsize with buffers eviction and initialization. From what I recall
> > > the size of buffer lookup table is about two orders of magnitude lower
> > > than shared buffers, so the overhead should not be that large even for
> > > significant amount of buffers.
> >
> > The proposal will work but will require significant work:
> >
> > 1. The pointer to the shared buffer lookup table will change.
>
> Which pointers you mean? AFAICT no operation on the buffer lookup table
> returns a pointer (they work with buffer id or a hash) and keys are
> compared by value as well.

The buffer lookup table itself.
/* Pass location of hashtable header to hash_create */
infoP->hctl = (HASHHDR *) location;

>
> > we can not have few processes accessing old lookup table and few
> > processes new one. That has potential to make many processes wait for
> > a very long time.
>
> As I've mentioned above, size of the buffer lookup table is few
> magnitudes lower than shared buffers, so I doubt about "a very long
> time". But it can be measured.
>
> > 2. The memory consumed by the old buffer lookup table will need to be
> > "freed" to the OS. The only way to do so is by having a new memory
> > segment
>
> Shared buffer lookup table already lives in it's own segment as
> implemented in the current patch, so I don't see any problem here.

The table is not a single chunk of memory. It's a few chunks spread
across the shared memory segment. Freeing a lookup table is like
freeing those chunks. We have ways to free tail parts of shared memory
segments, but not chunks in-between.

-- 
Best Wishes,
Ashutosh Bapat