Horton

Re: WIP: [[Parallel] Shared] Hash

Thomas Munro <thomas.munro@enterprisedb.com>

View thread View on postgr.es
From: Thomas Munro <thomas.munro@enterprisedb.com>
To: Pg Hackers <pgsql-hackers@postgresql.org>
Date: 2016-11-01T04:33:24Z
Lists: pgsql-hackers

Attachments

Thomas Munro <thomas.munro@enterprisedb.com> wrote:
> The basic approach is visible and simple cases are working though, so
> I am submitting this WIP work for a round of review in the current
> commitfest and hoping to get some feedback and ideas.  I will post the
> patch in a follow-up email shortly...

Aloha,

Please find a WIP patch attached.  Everything related to batch reading
is not currently in a working state, which breaks multi-batch joins,
but many single batch cases work correctly.  In an earlier version I
had multi-batch joins working but was before I started tackling
problems 2 and 3 listed in my earlier message.  There is some error
handling and resource cleanup missing, and doubtless some cases not
handled correctly.  But I thought it would be good to share this
development snapshot for discussion, so I'm posting this as is, and
will post an updated version when I've straightened out the batching
code some more.

To apply parallel-hash-v1, first apply the following patches, in this order:

condition-variable-v3.patch [1]
remove-useless-barrier-header-v2.patch [2]
barrier-v3.patch [2]
dsa-v4.patch [3]
dsa-area-for-executor-v1.patch [4]

When applying dsa-v4 on top of barrier-v3, it will reject a hunk in
src/backend/storage/ipc/Makefile where they both add their object
file.  Simply add dsa.o to OBJS manually.

Then you can apply parallel-hash-v1.patch, which is attached to this message.

[1] https://www.postgresql.org/message-id/flat/CA%2BTgmoaj2aPti0yho7FeEf2qt-JgQPRWb0gci_o1Hfr%3DC56Xng%40mail.gmail.com
[2] https://www.postgresql.org/message-id/CAEepm%3D1wrrzxh%3DSRCF_Hk4SZQ9BULy1vWsicx0EbgUf0B85vZQ%40mail.gmail.com
[3] https://www.postgresql.org/message-id/flat/CAEepm%3D1z5WLuNoJ80PaCvz6EtG9dN0j-KuHcHtU6QEfcPP5-qA%40mail.gmail.com
[4] https://www.postgresql.org/message-id/flat/CAEepm%3D0HmRefi1%2BxDJ99Gj5APHr8Qr05KZtAxrMj8b%2Bay3o6sA%40mail.gmail.com

-- 
Thomas Munro
http://www.enterprisedb.com

Commits

  1. Add shared tuplestores.

  2. Remove BufFile's isTemp flag.

  3. Provide DSM segment to ExecXXXInitializeWorker functions.

  4. Optimize joins when the inner relation can be proven unique.

  5. Fix failure to use clamp_row_est() for parallel joins.

  6. Document lack of validation when attaching foreign partitions.

  7. Fix inclusions of c.h from .h files.

  8. Fix inclusions of postgres_fe.h from .h files.

  9. Bring plpgsql into line with header inclusion policy.

  10. Document intentional violations of header inclusion policy.

  11. Shut down Gather's children before shutting down Gather itself.

  12. btree: Support parallel index scans.

  13. Add explicit ORDER BY to a few tests that exercise hash-join code.

  14. Revise hash join code so that we can increase the number of batches

  15. Rewrite hash join to use simple linked lists instead of a