Horton

Re: Extended Statistics set/restore/clear functions.

jian he <jian.universality@gmail.com>

View thread
From: jian he <jian.universality@gmail.com>
To: Corey Huinker <corey.huinker@gmail.com>
Cc: Michael Paquier <michael@paquier.xyz>, Tomas Vondra <tomas@vondra.me>, pgsql-hackers@lists.postgresql.org, tgl@sss.pgh.pa.us
Date: 2025-11-21T07:16:26Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add test doing some cloning of extended statistics data

  2. Add test for pg_restore_extended_stats() with multiranges

  3. Add support for "mcv" in pg_restore_extended_stats()

  4. Include extended statistics data in pg_dump

  5. Add support for "dependencies" in pg_restore_extended_stats()

  6. Add test for MAINTAIN permission with pg_restore_extended_stats()

  7. Add pg_restore_extended_stats()

  8. Add routine to free MCVList

  9. Improve pg_clear_extended_stats() with incorrect relation/stats combination

  10. Add pg_clear_extended_stats()

  11. Introduce routines to validate and free MVNDistinct and MVDependencies

  12. Fix typo in stat_utils.c

  13. Move attribute statistics functions to stat_utils.c

  14. Improve error messages of input functions for pg_dependencies and pg_ndistinct

  15. Improve test output of extended statistics for ndistinct and dependencies

  16. Fix some compiler warnings

  17. Add input function for data type pg_dependencies

  18. Add input function for data type pg_ndistinct

  19. Rework output format of pg_dependencies

  20. Rework output format of pg_ndistinct

  21. Fix comments of output routines for pg_ndistinct and pg_dependencies

  22. Move code specific to pg_dependencies to new file

  23. Move code specific to pg_ndistinct to new file

  24. Document some structures in attribute_stats.c

  25. Fix FATAL message for invalid recovery timeline at beginning of recovery

Attachments

hi.

makeJsonLexContextCstringLen in v16-0001 and v16-0002
should use GetDatabaseEncoding()?

wondering, should we check the value of "degree" is between 0 to 1?


+ if ((natts < 1) || (natts > (STATS_MAX_DIMENSIONS - 1)))
+ {
+ errsave(parse->escontext,
+ errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("malformed pg_dependencies: \"%s\"", parse->str),
+ errdetail("The \"%s\" key must contain an array of at least %d "
+  " and no than %d elements.",
+  PG_DEPENDENCIES_KEY_ATTRIBUTES, 1, STATS_MAX_DIMENSIONS - 1));

DETAIL:  The "attributes" key must contain an array of at least 1  and
no than 7 elements.
if you check the error message carefully, there is one more extra
white space after "at least 1".


typedef struct MVDependency
{
    double        degree;            /* degree of validity (0-1) */
    AttrNumber    nattributes;    /* number of attributes */
    AttrNumber    attributes[FLEXIBLE_ARRAY_MEMBER];    /* attribute numbers */
} MVDependency;

SELECT '[{"attributes" : [2], "dependency" : 4, "degree":
"NaN"}]'::pg_dependencies;
SELECT '[{"attributes" : [2], "dependency" : 4, "degree":
"-inf"}]'::pg_dependencies;
SELECT '[{"attributes" : [2], "dependency" : 4, "degree":
"inf"}]'::pg_dependencies;

either error out or not, pg_dependencies.sql needs tests like the
above to test these special values.
json does not support special value like, "inf", "NaN", so we can
SELECT '[{"attributes" : [2], "dependency" : 4, "degree":
"-inf"}]'::pg_dependencies;
but can not
SELECT '[{"attributes" : [2], "dependency" : 4, "degree":
"-inf"}]'::pg_dependencies::text::pg_dependencies;

I found that numeric values behave the same with or without double quotes.
SELECT '[{"ndistinct" : "1", "attributes" : ["2","3"]}]'::pg_ndistinct;
SELECT '[{"ndistinct" : 1, "attributes" : [2,3]}]'::pg_ndistinct;
SELECT '[{"attributes" : [1, 2], "dependency" : 4, "degree":
"1"}]'::pg_dependencies;
SELECT '[{"attributes" : [1, "2"], "dependency" : "4", "degree":
"1"}]'::pg_dependencies;
i guess the reason is in makeJsonLexContextCstringLen, we set
need_escapes to true.


I added more regress tests to improve coverage for
src/test/regress/sql/pg_dependencies.sql


SELECT '[{"attributes" : [-11,1], "dependency" : -1116, "degree":
"1.2"}]'::pg_dependencies;
DETAIL:  Invalid "attributes" element: -11.
we already check the key "attributes", we can also check "dependency",
IF the value of ""dependency"
does not meet
``(if (parse->dependency == 0 || parse->dependency < (0-STATS_MAX_DIMENSIONS))``
then error out.
see the attached patch.


--
jian
https://www.enterprisedb.com/