Horton

Re: [HACKERS] path toward faster partition pruning

David Rowley <david.rowley@2ndquadrant.com>

View thread View on postgr.es
From: David Rowley <david.rowley@2ndquadrant.com>
To: Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>
Cc: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>, Jesper Pedersen <jesper.pedersen@redhat.com>, Amit Langote <amitlangote09@gmail.com>, Rajkumar Raghuwanshi <rajkumar.raghuwanshi@enterprisedb.com>, Robert Haas <robertmhaas@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>, Beena Emerson <memissemerson@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2018-01-31T23:57:41Z
Lists: pgsql-hackers

Attachments

On 31 January 2018 at 21:03, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp> wrote:
> Update patch set attached.  Thanks again.

(My apologies for being slow to respond here. I've been on leave this
week and I'm off again next week. I now have a little time to reply)

Hi Amit,

Thanks for incorporating my changes into the patchset. A while ago I
was rebasing the run-time pruning patch on top of this but ran into a
few problems which are all results of my changes.

1. remove_redundant_clauses collapses the PartClause list into the
most restrictive set of clauses. This disallows multiple evaluations
of the PartitionClauseInfo during runtime pruning. I've written a
proposed fix for this and attached it.

2. PartitionClauseInfo->keyclauses is a list of PartClause which is
not a node type. This will cause _copyPartitionClauseInfo() to fail.

I'm still not quite sure the best way to fix #2 since PartClause
contains a FmgrInfo. I do have a local fix which moves PartClause to
primnodes.h and makes it a proper node type. I also added a copy
function which does not copy any of the cache fields in PartClause. It
just sets valid_cache to false. I didn't particularly think this was
the correct fix. I just couldn't think of how exactly this should be
done at the time.

The attached patch also adds the missing nodetag from
PartitionClauseInfo and also fixes up other code so as we don't memset
the node memory to zero, as that would destroy the tag. I ended up
just having extract_partition_key_clauses do the makeNode call. This
also resulted in populate_partition_clauses being renamed to
generate_partition_clauses

-- 
 David Rowley                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

Commits

  1. Fix assorted partition pruning bugs

  2. Make gen_partprune_steps static

  3. Remove useless 'default' clause

  4. Reorganize partitioning code

  5. Use custom hash opclass for hash partition pruning

  6. Blindly attempt to fix sepgsql tests broken due to 9fdb675fc5.

  7. Attempt to fix endianess issues in new hash partition test.

  8. Faster partition pruning

  9. For partitionwise join, match on partcollation, not parttypcoll.

  10. Revise API for partition bound search functions.

  11. Revise API for partition_rbound_cmp/partition_rbound_datum_cmp.

  12. Fix possible crash in partition-wise join.

  13. Refactor code for partition bound searching

  14. New C function: bms_add_range

  15. Add extensive tests for partition pruning.

  16. Add null test to partition constraint for default range partitions.

  17. Remove BufFile's isTemp flag.

  18. Make OWNER TO subcommand mention consistent

  19. Fix index matching for operators with mixed collatable/noncollatable inputs.