Horton

Re: Partition-wise join for join between (declaratively) partitioned tables

Thomas Munro <thomas.munro@enterprisedb.com>

View thread View on postgr.es
From: Thomas Munro <thomas.munro@enterprisedb.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Ashutosh Bapat <ashutosh.bapat@enterprisedb.com>, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>, Rafia Sabih <rafia.sabih@enterprisedb.com>, Rajkumar Raghuwanshi <rajkumar.raghuwanshi@enterprisedb.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2017-09-16T02:41:59Z
Lists: pgsql-hackers

Attachments

On Sat, Sep 16, 2017 at 9:38 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:
> On Sat, Sep 16, 2017 at 9:23 AM, Robert Haas <robertmhaas@gmail.com> wrote:
>> On the overall patch set:
>>
>> - I am curious to know how this has been tested.  How much of the new
>> code is covered by the tests in 0007-Partition-wise-join-tests.patch?
>> How much does coverage improve with
>> 0008-Extra-testcases-for-partition-wise-join-NOT-FOR-COMM.patch?  What
>> code, if any, is not covered by either of those test suites?  Could we
>> do meaningful testing of this with something like Andreas
>> Seltenreich's sqlsmith?
>
> FWIW I'm working on an answer to both of those question, but keep
> getting distracted by other things catching on fire...

I cobbled together some scripts to figure out the test coverage of
lines actually modified by this patch set.   Please see attached.

I'm not sure if there is an established or better way to do this, but
I used git-blame to figure out which lines of gcov output can be
blamed on Ashutosh and prepended that to the lines of gcov's output.
That allowed me to find new/changed code not covered by "make check".
I found 94 untested new lines with 0007 applied and 88 untested new
lines with 0008 applied.  The 6 lines that 0008 reaches and 0007
doesn't are:

 ======== src/backend/optimizer/path/allpaths.c ========
-[TOUCHED BY PATCH SET]     #####: 3303: mark_dummy_rel(rel);
-[TOUCHED BY PATCH SET]     #####: 3304: return;
-[TOUCHED BY PATCH SET]     #####: 1515: continue;
-[TOUCHED BY PATCH SET]     #####: 1526: continue;
 ======== src/backend/optimizer/util/pathnode.c ========
-[TOUCHED BY PATCH SET]     #####: 3433: break;
-[TOUCHED BY PATCH SET]     #####: 3435: return NULL;

-- 
Thomas Munro
http://www.enterprisedb.com

Commits

  1. Add test for partitionwise join involving default partition.

  2. Rewrite the code that applies scan/join targets to paths.

  3. Fix code related to partitioning schemes for dropped columns.

  4. Copy information from the relcache instead of pointing to it.

  5. Basic partition-wise join functionality.

  6. Associate partitioning information with each RelOptInfo.

  7. Expand partitioned table RTEs level by level, without flattening.

  8. Set partitioned_rels appropriately when UNION ALL is used.

  9. Remove dedicated B-tree root-split record types.

  10. Assorted preparatory refactoring for partition-wise join.

  11. Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.

  12. Avoid unnecessary single-child Append nodes.

  13. Revisit handling of UNION ALL subqueries with non-Var output columns.