Horton

Re: Proposal: Limitations of palloc inside checkpointer

Alexander Korotkov <aekorotkov@gmail.com>

View thread View on postgr.es
From: Alexander Korotkov <aekorotkov@gmail.com>
To: Xuneng Zhou <xunengzhou@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Heikki Linnakangas <hlinnaka@iki.fi>, Maxim Orlov <orlovmg@gmail.com>, Andres Freund <andres@anarazel.de>, Ekaterina Sokolova <e.sokolova@postgrespro.ru>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2025-08-07T13:32:41Z
Lists: pgsql-hackers 
On Thu, Aug 7, 2025 at 4:01 PM Xuneng Zhou <xunengzhou@gmail.com> wrote:
> On Thu, Aug 7, 2025 at 3:16 PM Xuneng Zhou <xunengzhou@gmail.com> wrote:
> >
> > Hi, Tom!
> >
> > Thanks for catching this.
> >
> > On Thu, Aug 7, 2025 at 2:58 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > >
> > > Alexander Korotkov <aekorotkov@gmail.com> writes:
> > > > I'm going to push this if no objections.
> > >
> > > I looked at these patches while preparing release notes, and
> > > found an oversight.  CheckpointerShmemInit does
> > >
> > >         CheckpointerShmem->max_requests = Min(NBuffers, MAX_CHECKPOINT_REQUESTS);
> > >
> > > but CheckpointerShmemSize still does
> > >
> > >     size = add_size(size, mul_size(NBuffers, sizeof(CheckpointerRequest)));
> > >
> > > So if NBuffers is more than MAX_CHECKPOINT_REQUESTS, we will allocate
> > > extra CheckpointerRequest array entries that we will never use,
> > > wasting shared memory.  Admittedly the amount is small relative to the
> > > shared buffers themselves, but at the very least this is confusing.
> > >
> > > The comment in CheckpointerShmemSize needs adjustment, too.
> >
> > I attached a patch to fix it.
>
> Sorry for the error in this patch. I forgot to take the min of
> NBuffers and MAX_CHECKPOINT_REQUESTS for mem size calculation.
> Many thanks to Alexander for correcting and pushing it.

No problem.  Thank you for the patch.  And thanks to Tom for catching this.

------
Regards,
Alexander Korotkov
Supabase

Commits

  1. Fix checkpointer shared memory allocation

  2. Limit checkpointer requests queue size

  3. Process sync requests incrementally in AbsorbSyncRequests