Horton

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

Amit Kapila <amit.kapila16@gmail.com>

View thread
From: Amit Kapila <amit.kapila16@gmail.com>
To: Alexander Korotkov <a.korotkov@postgrespro.ru>
Cc: Andres Freund <andres@anarazel.de>, konstantin knizhnik <k.knizhnik@postgrespro.ru>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2018-06-06T13:33:47Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Allow page lock to conflict among parallel group members.

  2. Allow relation extension lock to conflict among parallel group members.

  3. Add assert to ensure that page locks don't participate in deadlock cycle.

  4. Assert that we don't acquire a heavyweight lock on another object after

  5. Fix unsafe usage of strerror(errno) within ereport().

On Tue, Jun 5, 2018 at 7:35 PM, Alexander Korotkov <
a.korotkov@postgrespro.ru> wrote:

> On Tue, Jun 5, 2018 at 4:02 PM Andres Freund <andres@anarazel.de> wrote:
> > On 2018-06-05 13:09:08 +0300, Alexander Korotkov wrote:
> > > It appears that buffer replacement happening inside relation
> > > extension lock is affected by starvation on exclusive buffer mapping
> > > lwlocks and buffer content lwlocks, caused by many concurrent shared
> > > lockers.  So, fair lwlock patch have no direct influence to relation
> > > extension lock, which is naturally not even lwlock...
> >
> > Yea, that makes sense. I wonder how much the fix here is to "pre-clear"
> > a victim buffer, and how much is a saner buffer replacement
> > implementation (either by going away from O(NBuffers), or by having a
> > queue of clean victim buffers like my bgwriter replacement).
>
> The particular thing I observed on our environment is BufferAlloc()
> waiting hours on buffer partition lock.  Increasing NUM_BUFFER_PARTITIONS
> didn't give any significant help.  It appears that very hot page (root
> page of
> some frequently used index) reside on that partition, so this partition was
> continuously under shared lock.  So, in order to resolve without changing
> LWLock, we probably should move our buffers hash table to something
> lockless.
>
>
I think Robert's chash stuff [1] might be helpful to reduce the contention
you are seeing.


[1] -
https://www.postgresql.org/message-id/CA%2BTgmoYE4t-Pt%2Bv08kMO5u_XN-HNKBWtfMgcUXEGBrQiVgdV9Q%40mail.gmail.com

-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com