Horton

Re: 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

Merlin Moncure <mmoncure@gmail.com>

View thread
From: Merlin Moncure <mmoncure@gmail.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Sergey Koposov <koposov@ast.cam.ac.uk>, pgsql-hackers@postgresql.org
Date: 2012-05-24T19:24:41Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Improve bulk-insert performance by keeping the current target buffer pinned

On Thu, May 24, 2012 at 1:43 PM, Robert Haas <robertmhaas@gmail.com> wrote:
> On Thu, May 24, 2012 at 2:19 PM, Sergey Koposov <koposov@ast.cam.ac.uk> wrote:
>> On Thu, 24 May 2012, Robert Haas wrote:
>>> On Thu, May 24, 2012 at 1:42 PM, Sergey Koposov <koposov@ast.cam.ac.uk>
>>> wrote:
>>>>
>>>> I guess there is nothing catastrophically wrong with that, but still I'm
>>>> very suprised that you get severe locking problems (factor of two
>>>> slow-down)
>>>> when running parallel read-only transactions.
>>>
>>> Me, too.  How many concurrent connections are you running, and does
>>> your working set exceed shared_buffers?  Can you provide a
>>> self-contained reproducible test case?
>>
>> The last tests I've been doing were with 8 connections.
>> And the working set is roughly 30Gig, which is ~ 3x the shared buffers. (but
>> ~ 50% of RAM).
>
> Given that additional information, I would say these results are
> expected.  Unfortunately, our BufFreelistLock is a serious contention
> point, and I think that's what you're hitting.  See the graph here:
>
> http://rhaas.blogspot.com/2012/03/performance-and-scalability-on-ibm.html
>
> As you can see, raw performance isn't much worse with the larger data
> sets, but scalability at high connection counts is severely degraded
> once the working set no longer fits in shared_buffers.

Hm, wouldn't the BufFreelistLock issue be ameliorated if
StrategyGetBuffer could reserve multiple buffers so that you'd draw
down your local list and only then go back to the global pool? (easier
said than done obviously).

merlin