Horton

Re: [HACKERS] path toward faster partition pruning

Amit Langote <langote_amit_f8@lab.ntt.co.jp>

View thread View on postgr.es
From: Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Rajkumar Raghuwanshi <rajkumar.raghuwanshi@enterprisedb.com>, David Rowley <david.rowley@2ndquadrant.com>, Dilip Kumar <dilipbalaut@gmail.com>, Beena Emerson <memissemerson@gmail.com>, Pg Hackers <pgsql-hackers@postgresql.org>
Date: 2017-11-30T01:43:24Z
Lists: pgsql-hackers 
On 2017/11/30 7:15, Robert Haas wrote:
> On Wed, Nov 29, 2017 at 3:28 PM, Robert Haas <robertmhaas@gmail.com> wrote:
>> On Wed, Nov 22, 2017 at 3:59 AM, Amit Langote
>> <Langote_Amit_f8@lab.ntt.co.jp> wrote:
>>> It seems I wrote an Assert in the code to support hash partitioning that
>>> wasn't based on a valid assumption.  I was wrongly assuming that all hash
>>> partitions for a given modulus (largest modulus) must exist at any given
>>> time, but that isn't the case.
>>
>> Committed 0003 with some adjustments:
>>
>> * Renamed the new test to partition_prune.
>> * Moved the test to what I thought was a better place in the schedule
>> file, and made it consistent between serial_schedule and
>> parallel_schedule.
>> * commutates -> commuted
>> * removed wrong /* empty */ comment
>> * Updated expected output.  It surprised me a bit that the tests
>> weren't passing as you had them, but the differences I got - all
>> related to mc3p_default - seemed correct to me
> 
> Committed 0004 after reviewing the code and testing that it seems to
> work as advertised.

Thank you.

> 0005 looks like it might need to be split into smaller patches.  More
> broadly, the commit messages you wrote for for 0005, 0006, and 0008
> don't seem to me to do a great job explaining the motivation for the
> changes which they make.  They tell me what the patches do, but not
> why they are doing it.  If there's an email in this thread that
> explains that stuff, please point me to it and I'll go back and reread
> it more carefully; if not, I think I definitely need some more
> explanation both of the mission of each patch and the reason why the
> patch set is divided up in the way that it is.

I'm working on a revised version of these patches to address recent
comments by Horiguchi-san.  I will also consider the points above before
sending the new version.

Thanks,
Amit

Commits

  1. Fix assorted partition pruning bugs

  2. Make gen_partprune_steps static

  3. Remove useless 'default' clause

  4. Reorganize partitioning code

  5. Use custom hash opclass for hash partition pruning

  6. Blindly attempt to fix sepgsql tests broken due to 9fdb675fc5.

  7. Attempt to fix endianess issues in new hash partition test.

  8. Faster partition pruning

  9. For partitionwise join, match on partcollation, not parttypcoll.

  10. Revise API for partition bound search functions.

  11. Revise API for partition_rbound_cmp/partition_rbound_datum_cmp.

  12. Fix possible crash in partition-wise join.

  13. Refactor code for partition bound searching

  14. New C function: bms_add_range

  15. Add extensive tests for partition pruning.

  16. Add null test to partition constraint for default range partitions.

  17. Remove BufFile's isTemp flag.

  18. Make OWNER TO subcommand mention consistent

  19. Fix index matching for operators with mixed collatable/noncollatable inputs.