Re: heavily contended lwlocks with long wait queues scale badly

Jonathan S. Katz <jkatz@postgresql.org>

From: "Jonathan S. Katz" <jkatz@postgresql.org>
To: Michael Paquier <michael@paquier.xyz>, Nathan Bossart <nathandbossart@gmail.com>
Cc: Andres Freund <andres@anarazel.de>, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>, pgsql-hackers@postgresql.org, Thomas Munro <thomas.munro@gmail.com>, Yura Sokolov <y.sokolov@postgrespro.ru>
Date: 2024-01-11T14:47:33Z
Lists: pgsql-hackers
On 1/10/24 10:45 PM, Michael Paquier wrote:
> On Wed, Jan 10, 2024 at 09:17:47PM -0600, Nathan Bossart wrote:
>> Now that commit a4adc31 has had some time to bake and concerns about
>> unintended consequences may have abated, I wanted to revive this
>> back-patching discussion.  I see a few possibly-related reports [0] [1]
>> [2], and I'm now seeing this in the field, too.  While it is debatable
>> whether this is a bug, it's a quite nasty issue for users, and it's both
>> difficult to detect and difficult to work around.
> 
> +1, I've seen this becoming a PITA for a few things.  Knowing that the
> size of PGPROC does not change at all, I would be in favor for a
> backpatch, especially since it's been in the tree for more than 1
> year, and even more knowing that we have 16 released with this stuff
> in.

I have similar data sources to Nathan/Michael and I'm trying to avoid 
piling on, but one case that's interesting occurred after a major 
version upgrade from PG10 to PG14 on a database supporting a very 
active/highly concurrent workload. On inspection, it seems like 
backpatching would help this particularly case.

With 10/11 EOL, I do wonder if we'll see more of these reports on 
upgrade to < PG16.

(I was in favor of backpatching prior; opinion is unchanged).

Thanks,

Jonathan

Commits

  1. lwlock: Fix quadratic behavior with very long wait lists