Horton

RE: Changing shared_buffers without restart

Jack Ng <jack.ng@huawei.com>

View thread
From: Jack Ng <Jack.Ng@huawei.com>
To: Dmitry Dolgov <9erthalion6@gmail.com>
Cc: Andres Freund <andres@anarazel.de>, Thom Brown <thom@linux.com>, "Ashutosh Bapat" <ashutosh.bapat.oss@gmail.com>, Tomas Vondra <tomas@vondra.me>, "Thomas Munro" <thomas.munro@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>, Ni Ku <jakkuniku@gmail.com>
Date: 2025-07-15T22:52:01Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Remove PG_MMAP_FLAGS from mem.h

  2. Improve runtime and output of tests for replication slots checkpointing.

  3. Revert support for improved tracking of nested queries

  4. Use exported symbols list on macOS for loadable modules as well

  5. Add support for basic NUMA awareness

  6. Avoid unnecessary copying of a string in pg_restore.c

  7. aio: Infrastructure for io_method=worker

  8. Improve InitShmemAccess() prototype

>> On Mon, Jul 14, 2025 at 03:18:10PM +0000, Jack Ng wrote:
>> Just brain-storming here... would moving NBuffers to shared memory solve
>this specific issue? Though I'm pretty sure that would open up a new set of
>synchronization issues elsewhere, so I'm not sure if there's a net gain.
>
>It's in fact already happening, there is a shared structure that described the
>resize status. But if I get everything right, it doesn't solve all the problems.

Hi Dmitry, 

Just to clarify, you're not only referring to the ShmemControl::NSharedBuffers
and related logic in the current patches, but actually getting rid of per-process
NBuffers completely and use ShmemControl::NSharedBuffers everywhere instead (or
something along those lines)? So that when the coordinator updates
ShmemControl::NSharedBuffers, everyone sees the new value right away.
I guess this is part of the "simplified design" you mentioned several posts earlier?

I also thought about that approach more, and there seems to be new synchronization
issues we would need to deal with, like:

1. Mid-execution change of NBuffers in functions like BufferSync and BgBufferSync,
which could cause correctness and performance issues. I suppose most of them
are solvable with atomics and shared r/w locks etc, but at the cost of higher
performance overheads.

2. NBuffers becomes inconsistent with the underlying shared memory mappings for a
period of time for each process. Currently both are updated in AnonymousShmemResize
and AdjustShmemSize "atomically" for a process, so I wonder if letting them get
out-of-sync (even for a brief period) could be problematic.

I agree it doesn't seem to solve all the problems. It can simplify certain aspects
of the design, but may also introduce new issues. Overall not a "silver bullet" :)

Jack