Horton

Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

David Rowley <david.rowley@2ndquadrant.com>

View thread View on postgr.es
From: David Rowley <david.rowley@2ndquadrant.com>
To: Amit Langote <amitlangote09@gmail.com>
Cc: Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>, Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>, Rajkumar Raghuwanshi <rajkumar.raghuwanshi@enterprisedb.com>, Robert Haas <robertmhaas@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>, Beena Emerson <memissemerson@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2018-01-18T03:14:51Z
Lists: pgsql-hackers

Attachments

On 18 January 2018 at 00:13, David Rowley <david.rowley@2ndquadrant.com> wrote:
> On 17 January 2018 at 23:48, Amit Langote <amitlangote09@gmail.com> wrote:
>> I'm concerned that after your patch to remove
>> match_clauses_to_partkey(), we'd be doing more work than necessary in
>> some cases.  For example, consider the case of using run-time pruning
>> for nested loop where the inner relation is a partitioned table.  With
>> the old approach, get_partitions_from_clauses() would only be handed
>> the clauses that are known to match the partition keys (which most
>> likely is fewer than all of the query's clauses), so
>> get_partitions_from_clauses() doesn't have to do the work of filtering
>> non-partition clauses every time (that is, for every outer row).
>> That's why I had decided to keep that part in the planner.
>
> That might be better served by splitting
> classify_partition_bounding_keys() into separate functions, the first
> function would be in charge of building keyclauses_all. That way the
> remaining work during the executor would never need to match clauses
> to a partition key as they'd be in lists dedicated to each key.

I've attached another delta against your v20 patch which does this.
It's very rough for now and I've only checked that it passes the
regression test so far.

It will need some cleanup work, but I'd be keen to know what you think
of the general idea. I've not fully worked out how run-time pruning
will use this as it'll need another version of
get_partitions_from_clauses but passes in a PartScanClauseInfo
instead, and does not call extract_partition_key_clauses. That area
probably  needs some shuffling around so that does not end up a big
copy and paste of all that new logic.

-- 
 David Rowley                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

Commits

  1. Fix assorted partition pruning bugs

  2. Make gen_partprune_steps static

  3. Remove useless 'default' clause

  4. Reorganize partitioning code

  5. Use custom hash opclass for hash partition pruning

  6. Blindly attempt to fix sepgsql tests broken due to 9fdb675fc5.

  7. Attempt to fix endianess issues in new hash partition test.

  8. Faster partition pruning

  9. For partitionwise join, match on partcollation, not parttypcoll.

  10. Revise API for partition bound search functions.

  11. Revise API for partition_rbound_cmp/partition_rbound_datum_cmp.

  12. Fix possible crash in partition-wise join.

  13. Refactor code for partition bound searching

  14. New C function: bms_add_range

  15. Add extensive tests for partition pruning.

  16. Add null test to partition constraint for default range partitions.

  17. Remove BufFile's isTemp flag.

  18. Make OWNER TO subcommand mention consistent

  19. Fix index matching for operators with mixed collatable/noncollatable inputs.