Horton

Re: [HACKERS] path toward faster partition pruning

Amit Langote <langote_amit_f8@lab.ntt.co.jp>

View thread View on postgr.es
From: Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>
To: David Rowley <david.rowley@2ndquadrant.com>
Cc: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>, Rajkumar Raghuwanshi <rajkumar.raghuwanshi@enterprisedb.com>, Robert Haas <robertmhaas@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>, Beena Emerson <memissemerson@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2017-12-22T00:57:45Z
Lists: pgsql-hackers 
David

On 2017/12/21 21:04, David Rowley wrote:
> On 21 December 2017 at 23:38, Amit Langote
> <Langote_Amit_f8@lab.ntt.co.jp> wrote:
>> Attached updated set of patches.
> 
> Looks like the new not equals code does not properly take into account
> a missing NULL partition.
> 
> create table ta (a int not null) partition by list (a);
> create table ta1 partition of ta for values in(1,2);
> create table ta2 partition of ta for values in(3,4);
> explain select * from ta where a <> 1 and a <> 2;
> ERROR:  negative bitmapset member not allowed
> 
> -- Add null partition
> create table ta_null partition of ta for values in(null);
> explain select * from ta where a <> 1 and a <> 2; -- works now.
>                          QUERY PLAN
> -------------------------------------------------------------
>  Append  (cost=0.00..48.25 rows=2525 width=4)
>    ->  Seq Scan on ta2  (cost=0.00..48.25 rows=2525 width=4)
>          Filter: ((a <> 1) AND (a <> 2))
> (3 rows)
> 
> This code appears to be at fault:
> 
> /*
> * Also, exclude the "null-only" partition, because strict clauses in
> * ne_clauses will not select any rows from it.
> */
> if (count_partition_datums(relation, boundinfo->null_index) == 0)
> excluded_parts = bms_add_member(excluded_parts,
> boundinfo->null_index);

Oops, must check before going to count datums that a null-partition exists
at all.  Will post the fixed version shortly, thanks.

Regards,
Amit

Commits

  1. Fix assorted partition pruning bugs

  2. Make gen_partprune_steps static

  3. Remove useless 'default' clause

  4. Reorganize partitioning code

  5. Use custom hash opclass for hash partition pruning

  6. Blindly attempt to fix sepgsql tests broken due to 9fdb675fc5.

  7. Attempt to fix endianess issues in new hash partition test.

  8. Faster partition pruning

  9. For partitionwise join, match on partcollation, not parttypcoll.

  10. Revise API for partition bound search functions.

  11. Revise API for partition_rbound_cmp/partition_rbound_datum_cmp.

  12. Fix possible crash in partition-wise join.

  13. Refactor code for partition bound searching

  14. New C function: bms_add_range

  15. Add extensive tests for partition pruning.

  16. Add null test to partition constraint for default range partitions.

  17. Remove BufFile's isTemp flag.

  18. Make OWNER TO subcommand mention consistent

  19. Fix index matching for operators with mixed collatable/noncollatable inputs.