Re: [HACKERS] PATCH: multivariate histograms and MCV lists

Tomas Vondra <tomas.vondra@2ndquadrant.com>

From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Dean Rasheed <dean.a.rasheed@gmail.com>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>, Andres Freund <andres@anarazel.de>, Thomas Munro <thomas.munro@enterprisedb.com>, Mark Dilger <hornschnorter@gmail.com>, Adrien Nayrat <adrien.nayrat@dalibo.com>, Pg Hackers <pgsql-hackers@postgresql.org>
Date: 2018-03-27T17:34:26Z
Lists: pgsql-hackers
On 03/27/2018 07:03 PM, Dean Rasheed wrote:
> On 27 March 2018 at 14:58, Dean Rasheed <dean.a.rasheed@gmail.com> wrote:
>> On 27 March 2018 at 01:36, Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
>>> 4) handling of NOT clauses in MCV lists (and in histograms)
>>>
>>> The query you posted does not fail anymore...
>>>
>> Ah, it turns out the previous query wasn't actually failing for the
>> reason I thought it was -- it was failing because it had a
>> ScalarArrayOpExpr that was being passed to
>> mcv_clauselist_selectivity() because of the wrong list being passed to
>> it. I could see from the code that a NOT clause would have tripped it
>> up, but most NOT clauses actually get rewritten by negate_clause() so
>> they end up not being NOT clauses.
>>
> 
> Thinking about that some, I think that the only NOT clauses this needs
> to actually worry about are NOTs of boolean Vars. Anything else that
> this code supports will have been transformed into something other
> than a NOT before reaching this point. Thus it might be much simpler
> to handle that as a special case in statext_is_compatible_clause() and
> mcv_update_match_bitmap(), rather than trying to support general NOT
> clauses, and going through a recursive call to
> mcv_update_match_bitmap(), and then having to merge bitmaps. NOT of a
> boolean Var could then be treated just like var=false, setting the
> appropriate attribute match entry if it's found in the MCV list. This
> would allow clauses like (a=1 and NOT b) to be supported, which I
> don't think currently works, because fullmatch won't get set.
> 

Yes, I came to the same conclusion ;-) I'll send an updated patch later
today.

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


Commits

  1. Convert pre-existing stats_ext tests to new style

  2. Add support for multivariate MCV lists

  3. Improve ANALYZE's strategy for finding MCVs.

  4. Clone extended stats in CREATE TABLE (LIKE INCLUDING ALL)

  5. Try again to fix accumulation of parallel worker instrumentation.

  6. Adjust psql \d query to avoid use of @> operator.

  7. Message style fixes

  8. Add security checks to selectivity estimation functions