Horton

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Etsuro Fujita <etsuro.fujita@gmail.com>

View thread
From: Etsuro Fujita <etsuro.fujita@gmail.com>
To: amul sul <sulamul@gmail.com>
Cc: Thomas Munro <thomas.munro@gmail.com>, Robert Haas <robertmhaas@gmail.com>, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, Rajkumar Raghuwanshi <rajkumar.raghuwanshi@enterprisedb.com>, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>, Dmitry Dolgov <9erthalion6@gmail.com>, Thomas Munro <thomas.munro@enterprisedb.com>, Antonin Houska <ah@cybertec.at>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2019-07-08T11:33:32Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Suppress unused-variable warning.

  2. Allow partitionwise joins in more cases.

  3. Avoid crash in partitionwise join planning under GEQO.

  4. Disable support for partitionwise joins in problematic cases.

  5. Add plan_cache_mode setting

  6. Add test for partitionwise join involving default partition.

  7. Revise API for partition_rbound_cmp/partition_rbound_datum_cmp.

Attachments

On Wed, Jul 3, 2019 at 3:44 PM Etsuro Fujita <etsuro.fujita@gmail.com> wrote:
> On Tue, Jul 2, 2019 at 1:47 PM amul sul <sulamul@gmail.com> wrote:
> > Attached version is rebase atop of the latest master head(c74d49d41c), thanks.
>
> Thanks!  Will review.

I started reviewing this.  Here is my initial review comments:

* 0001-Hash-partition-bound-equality-refactoring-v22.patch

First of all, I agree with your view on hash partitioning:

"3. For hash partitioned tables however, we support partition-wise join
only when the bounds exactly match. For hash partitioning it's unusual
to have missing partitions and hence generic partition matching is not
required."

which is cited from the commit message for the main patch
"0002-Partition-wise-join-for-1-1-1-0-0-1-partition-matchi-v22.patch".
(I think it would be better if we can extend the partition matching to
the hash-partitioning case where there are missing partitions in
future, though.)  However, I don't think it's a good idea to do this
refactoring, because that would lead to duplicating the code to check
whether two given hash bound collections are equal in
partition_bounds_equal() and partition_hash_bounds_merge() that will
be added by the main patch, after all.  To avoid that, how about
calling partition_bounds_equal() from partition_hash_bounds_merge() in
the main patch, like the attached?  I also did some cleanup for
partition_hash_bounds_merge().  This change makes the refactoring
patch unnecessary, so I dropped it.  Also, I included the
regression-test patch
"0003-Tests-for-0-1-1-1-and-1-0-partition-matching-v22.patch" in the
attached, because make check didn't succeed without the
regression-test patch.

That's it for now.  I'll review the remaining parts (ie,
"0002-Partition-wise-join-for-1-1-1-0-0-1-partition-matchi-v22.patch"
and "0003-Tests-for-0-1-1-1-and-1-0-partition-matching-v22.patch")
closely next.

Best regards,
Etsuro Fujita