Horton

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

Robert Haas <robertmhaas@gmail.com>

View thread
From: Robert Haas <robertmhaas@gmail.com>
To: Mithun Cy <mithun.cy@enterprisedb.com>
Cc: Masahiko Sawada <sawada.mshk@gmail.com>, Andres Freund <andres@anarazel.de>, Tom Lane <tgl@sss.pgh.pa.us>, Thomas Munro <thomas.munro@enterprisedb.com>, Amit Kapila <amit.kapila16@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2018-01-04T16:39:40Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Allow page lock to conflict among parallel group members.

  2. Allow relation extension lock to conflict among parallel group members.

  3. Add assert to ensure that page locks don't participate in deadlock cycle.

  4. Assert that we don't acquire a heavyweight lock on another object after

  5. Fix unsafe usage of strerror(errno) within ereport().

On Tue, Jan 2, 2018 at 1:09 AM, Mithun Cy <mithun.cy@enterprisedb.com> wrote:
> So in case of  N_RELEXTLOCK_ENTS = 1 we can see regression as high 25%. ?

So now the question is: what do these results mean for this patch?

I think that the chances of someone simultaneously bulk-loading 16 or
more relations that all happen to hash to the same relation extension
lock bucket is pretty darn small.  Most people aren't going to be
running 16 bulk loads at the same time in the first place, and if they
are, then there's a good chance that at least some of those loads are
either actually to the same relation, or that many or all of the loads
are targeting the same filesystem and the bottleneck will occur at
that level, or that the loads are to relations which hash to different
buckets.  Now, if we want to reduce the chances of hash collisions, we
could boost the default value of N_RELEXTLOCK_ENTS to 2048 or 4096.

However, if we take the position that no hash collision probability is
low enough and that we must eliminate all chance of false collisions,
except perhaps when the table is full, then we have to make this
locking mechanism a whole lot more complicated.  We can no longer
compute the location of the lock we need without first taking some
other kind of lock that protects the mapping from {db_oid, rel_oid} ->
{memory address of the relevant lock}.  We can no longer cache the
location where we found the lock last time so that we can retake it.
If we do that, we're adding extra cycles and extra atomics and extra
code that can harbor bugs to every relation extension to guard against
something which I'm not sure is really going to happen.  Something
that's 3-8% faster in a case that occurs all the time and as much as
25% slower in a case that virtually never arises seems like it might
be a win overall.

However, it's quite possible that I'm not seeing the whole picture
here.  Thoughts?

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company