Horton

Re: [HACKERS] path toward faster partition pruning

David Rowley <david.rowley@2ndquadrant.com>

View thread View on postgr.es
From: David Rowley <david.rowley@2ndquadrant.com>
To: Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>
Cc: Jesper Pedersen <jesper.pedersen@redhat.com>, Robert Haas <robertmhaas@gmail.com>, Ashutosh Bapat <ashutosh.bapat@enterprisedb.com>, Rajkumar Raghuwanshi <rajkumar.raghuwanshi@enterprisedb.com>, Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>, Amit Langote <amitlangote09@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>, Beena Emerson <memissemerson@gmail.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2018-03-29T11:08:31Z
Lists: pgsql-hackers

Attachments

On 29 March 2018 at 21:35, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp> wrote:
> Beside fixing that, I have decided to get rid of the
> PartititionPruneStepOpNe (a special kind of base pruning step that was
> being used to prune list partitions using a set of <> operator clauses)
> and related functions.  Instead pruning for <> operator clauses is now
> implemented by using a combination of PartitionPruneStepOp and
> PartitionPruneStepCombine after adding a new combine op COMBINE_INVERT (I
> also renamed COMBINE_OR and COMBINE_AND to COMBINE_UNION and
> COMBINE_INTERSECT, respectively).  I decided to do so because the previous
> arrangement looked like a "hack" to support a special case that touched no
> less than quite a few places.

Hi Amit,

I've looked at the v44 patch. Thanks for making those changes.

The new not-equal handling code is not quite right.

DROP TABLE listp;
CREATE TABLE listp (a INT) PARTITION BY LIST(a);
CREATE TABLE listp1_3 PARTITION OF listp FOR VALUES IN(1,3);
CREATE TABLE listp_default PARTITION OF listp DEFAULT;

EXPLAIN SELECT * FROM listp WHERE a <> 1;
                            QUERY PLAN
------------------------------------------------------------------
 Append  (cost=0.00..54.56 rows=2537 width=4)
   ->  Seq Scan on listp1_3  (cost=0.00..41.88 rows=2537 width=4)
         Filter: (a <> 1)
(3 rows)

The default should be included here.

INSERT INTO listp VALUES(1),(2),(3);
SELECT * FROM listp WHERE a <> 1;
 a
---
 3
(1 row)

This code assumes its fine to just reverse the setting for default:

result->scan_default = !source->scan_default;

More complex handling is needed here.

I've attached a diff for a small set of other things I noticed while reviewing.

-- 
 David Rowley                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

Commits

  1. Fix assorted partition pruning bugs

  2. Make gen_partprune_steps static

  3. Remove useless 'default' clause

  4. Reorganize partitioning code

  5. Use custom hash opclass for hash partition pruning

  6. Blindly attempt to fix sepgsql tests broken due to 9fdb675fc5.

  7. Attempt to fix endianess issues in new hash partition test.

  8. Faster partition pruning

  9. For partitionwise join, match on partcollation, not parttypcoll.

  10. Revise API for partition bound search functions.

  11. Revise API for partition_rbound_cmp/partition_rbound_datum_cmp.

  12. Fix possible crash in partition-wise join.

  13. Refactor code for partition bound searching

  14. New C function: bms_add_range

  15. Add extensive tests for partition pruning.

  16. Add null test to partition constraint for default range partitions.

  17. Remove BufFile's isTemp flag.

  18. Make OWNER TO subcommand mention consistent

  19. Fix index matching for operators with mixed collatable/noncollatable inputs.