Horton

Re: pgsql: Add parallel-aware hash joins.

Thomas Munro <thomas.munro@enterprisedb.com>

View thread View on postgr.es
From: Thomas Munro <thomas.munro@enterprisedb.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Andres Freund <andres@anarazel.de>, pgsql-committers <pgsql-committers@postgresql.org>
Date: 2017-12-30T13:51:26Z
Lists: pgsql-hackers

Attachments

On Fri, Dec 29, 2017 at 2:21 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:
> On Thu, Dec 28, 2017 at 5:15 PM, Thomas Munro
> <thomas.munro@enterprisedb.com> wrote:
>> On Thu, Dec 28, 2017 at 3:32 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>> !                            Buckets: 1024 (originally 2048)  Batches: 1 (originally 1)  Memory Usage: 0kB
>>> !  Execution time: 243.120 ms
>>>
>>> I don't have enough insight to be totally sure what this means, but the
>>> "Memory Usage: 0kB" bit is obviously bogus, so I'd venture that at least
>>> part of the issue is failure to return stats from a worker.
>>
>> Hmm.  Yeah, that seems quite likely -- thanks.  Investigating now.
>
> This is explained by the early exit case in
> ExecParallelHashEnsureBatchAccessors().  With just the right timing,
> it finishes up not reporting the true nbatch number, and never calling
> ExecParallelHashUpdateSpacePeak().

Hi Tom,

You mentioned that prairiedog sees the problem about one time in
thirty.  Would you mind checking if it goes away with this patch
applied?

-- 
Thomas Munro
http://www.enterprisedb.com

Commits

  1. Update obsolete sentence in README.parallel.

  2. Rewrite ConditionVariableBroadcast() to avoid live-lock.

  3. Tweak parallel hash join test case in hopes of improving stability.

  4. Rename pg_rewind's copy_file_range() to avoid conflict with new linux syscall.

  5. Fix some minor errors in new PHJ code.

  6. Fix EXPLAIN ANALYZE output for Parallel Hash.

  7. Fix rare assertion failure in parallel hash join.

  8. Cancel CV sleep during subtransaction abort.

  9. Add parallel-aware hash joins.

  10. Fix EXPLAIN ANALYZE of hash join when the leader doesn't participate.

  11. Add some regression tests that exercise hash join code.