Horton

Re: pgsql: Add parallel-aware hash joins.

Andres Freund <andres@anarazel.de>

View thread View on postgr.es
From: Andres Freund <andres@anarazel.de>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Thomas Munro <thomas.munro@enterprisedb.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2018-01-24T18:28:41Z
Lists: pgsql-hackers 
Hi,

On 2018-01-24 13:11:22 -0500, Robert Haas wrote:
> So for me, the additional hash index tests don't cost anything
> measurable and the additional hash join tests cost about a second.  I
> think this probably accounts for why committers other than you keep
> "adding so much time to the regression tests".  On modern hardware,
> the costs just don't matter.

I very much agree with the general sentiment, but a second of a 25s test
certainly isn't nothing.  As I've just written a few messages upthread,
I think we can hide the overall timing costs to a much larger degree
than we're doing, but I don't think we need not pay attention at all.


> Now, how much should we care about the performance of software with a
> planned release date of 2018 on hardware discontinued in 2001,
> hardware that is apparently about 20 times slower than a modern
> laptop?  Some, perhaps, but maybe not a whole lot.  Removing tests
> that have found actual bugs because they cost runtime on ancient
> systems that nobody uses for serious work doesn't make sense to me.

I again agree with the sentiment. One caveat is that old machines also
somewhat approximate testing with more instrumentation / debugging
enabled (say valgrind, CLOBBER_CACHE_ALWAYS, etc). So removing excessive
test overhead has still quite some benefits. But I definitely do not
want to lower coverage to achieve it.

Greetings,

Andres Freund

Commits

  1. Update obsolete sentence in README.parallel.

  2. Rewrite ConditionVariableBroadcast() to avoid live-lock.

  3. Tweak parallel hash join test case in hopes of improving stability.

  4. Rename pg_rewind's copy_file_range() to avoid conflict with new linux syscall.

  5. Fix some minor errors in new PHJ code.

  6. Fix EXPLAIN ANALYZE output for Parallel Hash.

  7. Fix rare assertion failure in parallel hash join.

  8. Cancel CV sleep during subtransaction abort.

  9. Add parallel-aware hash joins.

  10. Fix EXPLAIN ANALYZE of hash join when the leader doesn't participate.

  11. Add some regression tests that exercise hash join code.