Horton

Re: scalability bottlenecks with (many) partitions (and more)

Tomas Vondra <tomas@vondra.me>

View thread
From: Tomas Vondra <tomas@vondra.me>
To: David Rowley <dgrowleyml@gmail.com>, Robert Haas <robertmhaas@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Andres Freund <andres@anarazel.de>
Date: 2024-09-04T15:37:24Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Make FP_LOCK_SLOTS_PER_BACKEND look like a function

  2. Fix asserts in fast-path locking code

  3. Increase the number of fast-path lock slots



On 9/4/24 17:12, David Rowley wrote:
> On Wed, 4 Sept 2024 at 03:06, Robert Haas <robertmhaas@gmail.com> wrote:
>>
>> On Mon, Sep 2, 2024 at 1:46 PM Tomas Vondra <tomas@vondra.me> wrote:
>>> But say we add a GUC and set it to -1 by default, in which case it just
>>> inherits the max_locks_per_transaction value. And then also provide some
>>> basic metric about this fast-path cache, so that people can tune this?
>>
>> All things being equal, I would prefer not to add another GUC for
>> this, but we might need it.
> 
> I think driving the array size from max_locks_per_transaction is a
> good idea (rounded up to the next multiple of 16?).

Maybe, although I was thinking we'd just use the regular doubling, to
get nice "round" numbers. It will likely overshoot a little bit (unless
people set the GUC to exactly 2^N), but I don't think that's a problem.

> If someone comes along one day and shows us a compelling case where
> some backend needs more than its fair share of locks and performance
> is bad because of that, then maybe we can consider adding a GUC then.
> Certainly, it's much easier to add a GUC later if someone convinces
> us that it's a good idea than it is to add it now and try to take it
> away in the future if we realise it's not useful enough to keep.
> 

Yeah, I agree with this.



regards

-- 
Tomas Vondra