Horton

Re: Proposal: Limitations of palloc inside checkpointer

Maxim Orlov <orlovmg@gmail.com>

View thread View on postgr.es
From: Maxim Orlov <orlovmg@gmail.com>
To: Ekaterina Sokolova <e.sokolova@postgrespro.ru>
Cc: pgsql-hackers@lists.postgresql.org
Date: 2025-02-26T08:46:45Z
Lists: pgsql-hackers

Attachments

On Tue, 25 Feb 2025 at 22:44, Ekaterina Sokolova <e.sokolova@postgrespro.ru>
wrote:

> Hi, hackers!
>
> Historically, the checkpointer process use palloc() into
> AbsorbSyncRequests() function. Therefore, the checkpointer does not
> expect to receive a request larger than 1 GB.


Yeah. And the most unpleasant thing is it won't simply fail with an error
or helpful message suggesting a workaround (reduce the amount of shared
memory). Checkpointer will just "stuck".

AFAICS, we have a few options:
1. Leave it as it is, but fatal on allocation of the chunk more than 1G.
2. Use palloc_extended with MCXT_ALLOC_HUGE flag.
3. Do not use any allocation and use CheckpointerShmem->requests directly
in case of > 1G size of the required allocation.

Case (3) is not an option, in my opinion. So, we following (1) or (2).
Personally, I'm for (2), PFA v0 patch.


-- 
Best regards,
Maxim Orlov.

Commits

  1. Fix checkpointer shared memory allocation

  2. Limit checkpointer requests queue size

  3. Process sync requests incrementally in AbsorbSyncRequests