Re: multivariate statistics / patch v6

Tomas Vondra <tomas.vondra@2ndquadrant.com>

From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Jeff Janes <jeff.janes@gmail.com>, Stephen Frost <sfrost@snowman.net>
Cc: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2015-05-06T20:03:50Z
Lists: pgsql-hackers

Attachments

Attached is v6 of the multivariate stats, with a number of improvements:

1) fix of the contrib compile-time errors (reported by Jeff)

2) fix of pg_proc issues (reported by Jeff)

3) rebase to current master

4) fix a bunch of issues in the previous patches, due to referencing
    some parts too early (e.g. histograms in the first patch, etc.)

5) remove the explicit DELETEs from pg_mv_statistic (in the regression
    tests), this is now handled automatically by DROP TABLE etc.

6) number of performance optimizations in selectivity estimations:

    (a) minimize calls to get_oprrest, significantly reducing
        syscache calls

    (b) significant reduction of palloc overhead in deserialization of
        MCV lists and histograms

    (c) use more compact serialized representation of MCV lists and
        histograms, often removing ~50% of the size

    (d) use histograms with limited deserialization, which also allows
        caching function calls

    (e) modified histogram bucket partitioning, resulting in more even
        bucket distribution (i.e. producing buckets with more equal
        density and about equal size of each dimension)

7) add functions for listing MCV list items and histogram buckets:

     - pg_mv_mcvlist_items(oid)
     - pg_mv_histogram_buckets(oid, type)

    This is quite useful when analyzing the MCV lists / histograms.

8) improved support for OR clauses

9) allow calling pull_varnos() on expression trees containing
    RestrictInfo nodes (not sure if this is the right fix, it's being
    discussed in another thread)



--
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Commits

  1. Collect and use multi-column dependency stats

  2. Implement SortSupport for macaddr data type

  3. Implement multivariate n-distinct coefficients

  4. Generate fmgr prototypes automatically