Horton

Re: Extended Statistics set/restore/clear functions.

Corey Huinker <corey.huinker@gmail.com>

View thread
From: Corey Huinker <corey.huinker@gmail.com>
To: Tomas Vondra <tomas@vondra.me>
Cc: pgsql-hackers@lists.postgresql.org
Date: 2025-01-23T20:52:55Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add test doing some cloning of extended statistics data

  2. Add test for pg_restore_extended_stats() with multiranges

  3. Add support for "mcv" in pg_restore_extended_stats()

  4. Include extended statistics data in pg_dump

  5. Add support for "dependencies" in pg_restore_extended_stats()

  6. Add test for MAINTAIN permission with pg_restore_extended_stats()

  7. Add pg_restore_extended_stats()

  8. Add routine to free MCVList

  9. Improve pg_clear_extended_stats() with incorrect relation/stats combination

  10. Add pg_clear_extended_stats()

  11. Introduce routines to validate and free MVNDistinct and MVDependencies

  12. Fix typo in stat_utils.c

  13. Move attribute statistics functions to stat_utils.c

  14. Improve error messages of input functions for pg_dependencies and pg_ndistinct

  15. Improve test output of extended statistics for ndistinct and dependencies

  16. Fix some compiler warnings

  17. Add input function for data type pg_dependencies

  18. Add input function for data type pg_ndistinct

  19. Rework output format of pg_dependencies

  20. Rework output format of pg_ndistinct

  21. Fix comments of output routines for pg_ndistinct and pg_dependencies

  22. Move code specific to pg_dependencies to new file

  23. Move code specific to pg_ndistinct to new file

  24. Document some structures in attribute_stats.c

  25. Fix FATAL message for invalid recovery timeline at beginning of recovery

>
>
>
> > * no negative attnums in key list
>

Disregard this suggestion - negative attnums mean the Nth expression in the
extended stats object, though it boggles the mind how we could have 222
expressions...



> > * no duplicate attnums in key list
>

This one is still live, am considering.

At this point I was really thinking only about validating the attnums,
> i.e. to make sure it's a valid attribute in the table / statistics. That
> is something the pg_set_attribute_stats() enforce too, thanks to having
> a separate argument for the attribute name.
>
> That's where I'd stop. I don't want to do checks on the statistics
> content, like verifying the frequencies in the MCV sum up to 1.0 or
> stuff like that. I think we're not doing that for pg_set_attribute_stats
>

Agreed.


> either (and I'd bet one could cause a lot of "fun" this way).
>

If by "fun" you mean "create a fuzzing tool", then yes.

As an aside, the "big win" in all these functions is the ability to dump a
database --no-data, but have all the schema and statistics, thus allowing
for checking query plans on existing databases with sensitive data while
not actually exposing the data (except mcv, obvs), nor spending the I/O to
load that data.


> Understood. IMHO it's fine to say we're not validating the statistics
> are "consistent" but I think we should check it matches the definition.
>

+1



> > I suppose someone could write the following utility functions
> >
> >     pg_xlat_ndistinct_to_attnames(relation reloid, ndist pg_ndistinct) -
> >> json
> >     pg_xlat_ndistinct_from_attnames(relation reloid, ndist json) ->
> > pg_ndistinct
> >
> > and that would bridge the gap for the special case where you want to
> > adapt pg_ndistinct from one table structure to a slightly different one.
> >
> >
>
> OK
>

As they'll be pure-SQL functions, I'll likely post the definitions here,
but not put them into a patch unless it draws interest.
 > For that matter, it might make sense to break out the expressions code

> > into its own file, because every other stat attribute has its own.
> > Thoughts on that?
> >
>
> +1 to that, if it reduced unnecessary code duplication
>

I'm uncertain that it actually would deduplicate any code, but I'll
certainly try.