Re: [HACKERS] PATCH: multivariate histograms and MCV lists

Mark Dilger <hornschnorter@gmail.com>

From: Mark Dilger <hornschnorter@gmail.com>
To: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Cc: Adrien Nayrat <adrien.nayrat@dalibo.com>, Pg Hackers <pgsql-hackers@postgresql.org>
Date: 2017-11-25T16:15:05Z
Lists: pgsql-hackers
> On Nov 18, 2017, at 12:28 PM, Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
> 
> Hi,
> 
> Attached is an updated version of the patch, adopting the psql describe
> changes introduced by 471d55859c11b.
> 
> regards
> 
> -- 
> Tomas Vondra                  http://www.2ndQuadrant.com
> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
> <0001-multivariate-MCV-lists.patch.gz><0002-multivariate-histograms.patch.gz>

Thanks, Tomas, again for your work on this feature.

Applying just the 0001-multivariate-MCV-lists.patch to the current master, and
then extending the stats_ext.sql test as follows, I am able to trigger an error,
"ERROR:  operator 4294934272 is not a valid ordering operator".

 

diff --git a/src/test/regress/sql/stats_ext.sql b/src/test/regress/sql/stats_ext.sql
index e9902ced5c..5083dc05e6 100644
--- a/src/test/regress/sql/stats_ext.sql
+++ b/src/test/regress/sql/stats_ext.sql
@@ -402,4 +402,22 @@ EXPLAIN (COSTS OFF)
 EXPLAIN (COSTS OFF)
  SELECT * FROM mcv_lists WHERE a IS NULL AND b IS NULL AND c IS NULL;
 
-RESET random_page_cost;
+DROP TABLE mcv_lists; 
+ 
+CREATE TABLE mcv_lists (
+    a NUMERIC[],
+   b NUMERIC[]
+);
+CREATE STATISTICS mcv_lists_stats (mcv) ON a, b FROM mcv_lists;
+INSERT INTO mcv_lists (a, b)
+   (SELECT array_agg(gs::numeric) AS a, array_agg(gs::numeric) AS b
+       FROM generate_series(1,1000) gs
+   );
+ANALYZE mcv_lists;
+INSERT INTO mcv_lists (a, b)
+   (SELECT array_agg(gs::numeric) AS a, array_agg(gs::numeric) AS b
+       FROM generate_series(1,1000) gs
+   );
+ANALYZE mcv_lists;
+
+DROP TABLE mcv_lists;

Which gives me the following regression.diffs:

*** /Users/mark/master/postgresql/src/test/regress/expected/stats_ext.out   2017-11-25 08:06:37.000000000 -0800
--- /Users/mark/master/postgresql/src/test/regress/results/stats_ext.out    2017-11-25 08:10:18.000000000 -0800
***************
*** 721,724 ****
           Index Cond: ((a IS NULL) AND (b IS NULL))
  (5 rows)
  
! RESET random_page_cost;
--- 721,741 ----
           Index Cond: ((a IS NULL) AND (b IS NULL))
  (5 rows)
  
! DROP TABLE mcv_lists;
! CREATE TABLE mcv_lists (
!     a NUMERIC[],
!   b NUMERIC[]
! );
! CREATE STATISTICS mcv_lists_stats (mcv) ON a, b FROM mcv_lists;
! INSERT INTO mcv_lists (a, b)
!   (SELECT array_agg(gs::numeric) AS a, array_agg(gs::numeric) AS b
!       FROM generate_series(1,1000) gs
!   );
! ANALYZE mcv_lists;
! INSERT INTO mcv_lists (a, b)
!   (SELECT array_agg(gs::numeric) AS a, array_agg(gs::numeric) AS b
!       FROM generate_series(1,1000) gs
!   );
! ANALYZE mcv_lists;
! ERROR:  operator 4294934272 is not a valid ordering operator
! DROP TABLE mcv_lists;

======================================================================



Commits

  1. Convert pre-existing stats_ext tests to new style

  2. Add support for multivariate MCV lists

  3. Improve ANALYZE's strategy for finding MCVs.

  4. Clone extended stats in CREATE TABLE (LIKE INCLUDING ALL)

  5. Try again to fix accumulation of parallel worker instrumentation.

  6. Adjust psql \d query to avoid use of @> operator.

  7. Message style fixes

  8. Add security checks to selectivity estimation functions