Horton

Re: Proposal: Limitations of palloc inside checkpointer

Alexander Korotkov <aekorotkov@gmail.com>

View thread View on postgr.es
From: Alexander Korotkov <aekorotkov@gmail.com>
To: Xuneng Zhou <xunengzhou@gmail.com>
Cc: Heikki Linnakangas <hlinnaka@iki.fi>, Maxim Orlov <orlovmg@gmail.com>, Andres Freund <andres@anarazel.de>, Ekaterina Sokolova <e.sokolova@postgrespro.ru>, pgsql-hackers@lists.postgresql.org
Date: 2025-06-02T19:03:14Z
Lists: pgsql-hackers 
Hi, Xuneng Zhou!

On Tue, Apr 15, 2025 at 7:02 AM Xuneng Zhou <xunengzhou@gmail.com> wrote:
> I've moved this entry to the July CommitFest. The CI reported an unused variable warning in patch v5, so I've addressed it by removing the unused one. Sorry for not catching the issue locally.

Thank you for your work on this subject!

I have few notes about that:
1) Should we make CompactCheckpointerRequestQueue() process the queue
of checkpoint requests in smaller parts for the same reason we do this
in AbsorbSyncRequests()? That would require significant redesign of
the algorithm, but still.
2) That's pretty independent to the changes by the patch, but should
CompactCheckpointerRequestQueue() fill the gaps with entries from the
tail instead of rewriting the whole queue?  That might be a bit
faster.
3) For sure, we wouldn't backpatch this.  Can we prepare some simple
solution for back branches?  Perhaps, just introduction of
MAX_CHECKPOINT_REQUESTS is enough to save us from allocations larger
than 1GB.

------
Regards,
Alexander Korotkov
Supabase

Commits

  1. Fix checkpointer shared memory allocation

  2. Limit checkpointer requests queue size

  3. Process sync requests incrementally in AbsorbSyncRequests