Re: [HACKERS] path toward faster partition pruning

Alvaro Herrera <alvherre@alvh.no-ip.org>

From: Alvaro Herrera <alvherre@alvh.no-ip.org>
To: Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>
Cc: David Rowley <david.rowley@2ndquadrant.com>, Amit Langote <amitlangote09@gmail.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2018-04-06T20:28:00Z
Lists: pgsql-hackers
So I pushed this 25 minutes ago, and already there's a couple of
buildfarm members complaining:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=quokka&dt=2018-04-06%2020%3A09%3A52
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=termite&dt=2018-04-06%2019%3A55%3A07

Both show exactly the same diff in test partition_prune:

*** /home/pgbuildfarm/buildroot-termite/HEAD/pgsql.build/../pgsql/src/test/regress/expected/partition_prune.out	Fri Apr  6 15:55:08 2018
--- /home/pgbuildfarm/buildroot-termite/HEAD/pgsql.build/src/test/regress/results/partition_prune.out	Fri Apr  6 16:01:40 2018
***************
*** 1348,1357 ****
  ----------+----+-----
   hp0      |    | 
   hp0      |  1 | 
!  hp0      |  1 | xxx
   hp3      | 10 | yyy
!  hp1      |    | xxx
!  hp2      | 10 | xxx
  (6 rows)
  
  -- partial keys won't prune, nor would non-equality conditions
--- 1348,1357 ----
  ----------+----+-----
   hp0      |    | 
   hp0      |  1 | 
!  hp0      | 10 | xxx
!  hp3      |    | xxx
   hp3      | 10 | yyy
!  hp2      |  1 | xxx
  (6 rows)
  
  -- partial keys won't prune, nor would non-equality conditions
***************
*** 1460,1466 ****
                     QUERY PLAN                    
  -------------------------------------------------
   Append
!    ->  Seq Scan on hp0
           Filter: ((a = 1) AND (b = 'xxx'::text))
  (3 rows)
  
--- 1460,1466 ----
                     QUERY PLAN                    
  -------------------------------------------------
   Append
!    ->  Seq Scan on hp2
           Filter: ((a = 1) AND (b = 'xxx'::text))
  (3 rows)
  
***************
*** 1468,1474 ****
                       QUERY PLAN                      
  -----------------------------------------------------
   Append
!    ->  Seq Scan on hp1
           Filter: ((a IS NULL) AND (b = 'xxx'::text))
  (3 rows)
  
--- 1468,1474 ----
                       QUERY PLAN                      
  -----------------------------------------------------
   Append
!    ->  Seq Scan on hp3
           Filter: ((a IS NULL) AND (b = 'xxx'::text))
  (3 rows)
  
***************
*** 1476,1482 ****
                      QUERY PLAN                    
  --------------------------------------------------
   Append
!    ->  Seq Scan on hp2
           Filter: ((a = 10) AND (b = 'xxx'::text))
  (3 rows)
  
--- 1476,1482 ----
                      QUERY PLAN                    
  --------------------------------------------------
   Append
!    ->  Seq Scan on hp0
           Filter: ((a = 10) AND (b = 'xxx'::text))
  (3 rows)
  
***************
*** 1494,1504 ****
   Append
     ->  Seq Scan on hp0
           Filter: (((a = 10) AND (b = 'yyy'::text)) OR ((a = 10) AND (b = 'xxx'::text)) OR ((a IS NULL) AND (b IS NULL)))
-    ->  Seq Scan on hp2
-          Filter: (((a = 10) AND (b = 'yyy'::text)) OR ((a = 10) AND (b = 'xxx'::text)) OR ((a IS NULL) AND (b IS NULL)))
     ->  Seq Scan on hp3
           Filter: (((a = 10) AND (b = 'yyy'::text)) OR ((a = 10) AND (b = 'xxx'::text)) OR ((a IS NULL) AND (b IS NULL)))
! (7 rows)
  
  -- hash partitiong pruning doesn't occur with <> operator clauses
  explain (costs off) select * from hp where a <> 1 and b <> 'xxx';
--- 1494,1502 ----
   Append
     ->  Seq Scan on hp0
           Filter: (((a = 10) AND (b = 'yyy'::text)) OR ((a = 10) AND (b = 'xxx'::text)) OR ((a IS NULL) AND (b IS NULL)))
     ->  Seq Scan on hp3
           Filter: (((a = 10) AND (b = 'yyy'::text)) OR ((a = 10) AND (b = 'xxx'::text)) OR ((a IS NULL) AND (b IS NULL)))
! (5 rows)
  
  -- hash partitiong pruning doesn't occur with <> operator clauses
  explain (costs off) select * from hp where a <> 1 and b <> 'xxx';


-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


Commits

  1. Fix assorted partition pruning bugs

  2. Make gen_partprune_steps static

  3. Remove useless 'default' clause

  4. Reorganize partitioning code

  5. Use custom hash opclass for hash partition pruning

  6. Blindly attempt to fix sepgsql tests broken due to 9fdb675fc5.

  7. Attempt to fix endianess issues in new hash partition test.

  8. Faster partition pruning

  9. For partitionwise join, match on partcollation, not parttypcoll.

  10. Revise API for partition bound search functions.

  11. Revise API for partition_rbound_cmp/partition_rbound_datum_cmp.

  12. Fix possible crash in partition-wise join.

  13. Refactor code for partition bound searching

  14. New C function: bms_add_range

  15. Add extensive tests for partition pruning.

  16. Add null test to partition constraint for default range partitions.

  17. Remove BufFile's isTemp flag.

  18. Make OWNER TO subcommand mention consistent

  19. Fix index matching for operators with mixed collatable/noncollatable inputs.