Horton

Re: Proposal: Limitations of palloc inside checkpointer

Xuneng Zhou <xunengzhou@gmail.com>

View thread View on postgr.es
From: Xuneng Zhou <xunengzhou@gmail.com>
To: Alexander Korotkov <aekorotkov@gmail.com>
Cc: Heikki Linnakangas <hlinnaka@iki.fi>, Maxim Orlov <orlovmg@gmail.com>, Andres Freund <andres@anarazel.de>, Ekaterina Sokolova <e.sokolova@postgrespro.ru>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2025-06-04T12:33:30Z
Lists: pgsql-hackers 
Hi,

Thanks for the feedback!

> I think it would be good start point to use the same batch size of
> CompactCheckpointerRequestQueue() and AbsorbSyncRequests()
>

So we’ll keep both batch processing and the request cap in place for now.

> > > Right, but another point is to avoid lengthy holding of
> > > CheckpointerCommLock.  What do you think about that?
> >
> > I am not clear on this. Could you elaborate on it?
>
> See [1] for more detailed description of this.

> Links.
> 1. https://www.postgresql.org/message-id/flat/db4534f83a22a29ab5ee2566ad86ca92%40postgrespro.ru

I read the thread but didn’t find a specific explanation of avoiding
long lock holds. My understanding is: when compaction processes a very
large queue in one go, it holds CheckpointerCommLock the entire time,
blocking all ForwardSyncRequest callers. Batch processing would
release the lock after each chunk, allowing other backends to make
progress. Is that correct?

Commits

  1. Fix checkpointer shared memory allocation

  2. Limit checkpointer requests queue size

  3. Process sync requests incrementally in AbsorbSyncRequests