Re: [HACKERS] PATCH: multivariate histograms and MCV lists
Tomas Vondra <tomas.vondra@2ndquadrant.com>
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Dean Rasheed <dean.a.rasheed@gmail.com>
Cc: Thomas Munro <thomas.munro@enterprisedb.com>,
Bruce Momjian <bruce@momjian.us>, Alvaro Herrera <alvherre@2ndquadrant.com>,
Andres Freund <andres@anarazel.de>, Mark Dilger <hornschnorter@gmail.com>,
Pg Hackers <pgsql-hackers@postgresql.org>
Date: 2019-01-14T19:21:45Z
Lists: pgsql-hackers
Attachments
- 0001-multivariate-MCV-lists-20190114.patch.gz (application/gzip) patch 0001
- 0002-multivariate-histograms-20190114.patch.gz (application/gzip) patch 0002
On 1/14/19 4:31 PM, Tomas Vondra wrote:
>
> On 1/14/19 12:20 PM, Dean Rasheed wrote:
>> (Removing Adrien from the CC list, because messages to that address
>> keep bouncing)
>>
>> On Sun, 13 Jan 2019 at 00:31, Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
>>>
>>> On 1/10/19 6:09 PM, Dean Rasheed wrote:
>>>>
>>>> In the previous discussion around UpdateStatisticsForTypeChange(), the
>>>> consensus appeared to be that we should just unconditionally drop all
>>>> extended statistics when ALTER TABLE changes the type of an included
>>>> column (just as we do for per-column stats), since such a type change
>>>> can rewrite the data in arbitrary ways, so there's no reason to assume
>>>> that the old stats are still valid. I think it makes sense to extract
>>>> that as a separate patch to be committed ahead of these ones, and I'd
>>>> also argue for back-patching it.
>>>
>>> Wasn't the agreement to keep stats that don't include column values
>>> (functional dependencies and ndistinct coefficients), and reset only
>>> more complex stats? That's what happens in master and how it's extended
>>> by the patch for MCV lists and histograms.
>>>
>>
>> Ah OK, I misremembered the exact conclusion reached last time. In that
>> case the logic in UpdateStatisticsForTypeChange() looks wrong:
>>
>> /*
>> * If we can leave the statistics as it is, just do minimal cleanup
>> * and we're done.
>> */
>> if (!attribute_referenced && reset_stats)
>> {
>> ReleaseSysCache(oldtup);
>> return;
>> }
>>
>> That should be "|| !reset_stats", or have more parentheses.
>
> Yeah, it should have been
>
> if (!(attribute_referenced && reset_stats))
>
> i.e. there's a parenthesis missing. Thanks for noticing this. I guess a
> regression test for this would be useful.
>
>> In fact, I think that computing attribute_referenced is unnecessary
>> because the dependency information includes the columns that the
>> stats are for and ATExecAlterColumnType() uses that, so
>> attribute_referenced will always be true.
> Hmmm. I'm pretty sure I came to the conclusion it's in fact necessary,
> but I might be wrong. Will check.
>
Turns out you were right - the attribute_referenced piece was quite
unnecessary. So I've removed it. I've also extended the regression tests
to verify changing type of another column does not reset the stats.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Commits
-
Convert pre-existing stats_ext tests to new style
- dbb984128ebf 12.0 landed
-
Add support for multivariate MCV lists
- 7300a699502f 12.0 landed
-
Improve ANALYZE's strategy for finding MCVs.
- b5db1d93d2a6 11.0 cited
-
Clone extended stats in CREATE TABLE (LIKE INCLUDING ALL)
- 5564c1181548 11.0 cited
-
Try again to fix accumulation of parallel worker instrumentation.
- 8526bcb2df76 11.0 cited
-
Adjust psql \d query to avoid use of @> operator.
- 471d55859c11 11.0 cited
-
Message style fixes
- 821fb8cdbf70 11.0 cited
-
Add security checks to selectivity estimation functions
- e2d4ef8de869 10.0 cited