Re: Showing applied extended statistics in explain Part 2

Hi All,

I've organized the discussion so far and improved the patch.
The issues and their status are below.

* Issues and status (or comment):

I've numbered them for ease of management.
====
T6. Changed option to show extended statistics (from VERBOSE to STATS)
    -> Done already on the previous patch

T4. Add new node (to resolve errors in cfbot and prepared statement)
    -> Done on the attached patch maybe

M2. Handle VERBOSE option more correctly
    -> Done on the attache patch

M4. Organize the specifications for the types of ExtStats to display
    -> Done. I understood the specifications (restrictions) below and wrote
it
       on explain.sgml.
        - depndencied or mcv of extended stats are displayed
        - ndistinct is excluded

T9. Add log_stats option to auto_explain
    -> Done on the attached patches

I2. Add tab completion to psql
    -> Done on the attached patches

T7. Add documents
    -> Partially completed.
        - Done:
            - doc/src/sgml/ref/explain.sgml
            - doc/src/sgml/auto-explain.sgml
        - Not started yet:
            - doc/src/sgml/ref/create_statistics.sgml
            - doc/src/sgml/perform.sgml
            - doc/src/sgml/planstats.sgml

T8. Add regression test (stats_ext.sql)
    -> Work in progress.

M5. Extstats showing twice issue
    -> Work in progress.
       The problem is related to Partial/Final groupby or Parallel query.
       I found that HashParallel is one of the causes to show it twice.
       (See result.txt)

I1. Fix error when query has nested OR condition
    -> Not started yet.
       As a starting point, it's good to understand the difference
       between maybe_extract_actual_clauses() and extract_actual_clauses(),
right?

T5. Measure overhead of the feature
    -> Not started yet.
       But I understood a conditions to measure overhead:
          - simple query without no extended stats
          - simple query with 100 extended stats
====


* Regarding to the attached patches:

This patch is WIP patch including the following fixes:
====
- 0001-Add-a-new-option-STATS-to-EXPLAIN-command-r3.patch
    - Rebased on 9926f854
    - Added a new struct Applied_ExtStats in plannode.h (T4)
        - Hopefully this will solve the issue related to
"-DCOPY_PARSE_PLAN_TREES"
          with cfbot.
        - To pass the extended statistics information from path to plan,
          it might be more appropriate to define a new structure in
primnode.h
          rather than plannode.h. Any advice would be appreciated.
    - Handled EXPLAIN(STATS, VERBOSE) option (M2)
        - Before the fix, schema name was always added to extended
statistics name,
          but with this patch, schema name is added to the following only
when
          VERBOSE option is selected:
            - Extended statistics name, table name, and column name
    - Added Supported extended statistics types in document (M4)

- 0002-Add-a-new-option-auto_explain.log_stats.patch
    - It allows to use auto_explain.log_stats option on auto_explain

- 0003-Add-a-new-tab-completion-for-EXPLAIN-STATS-on-psql.patch
    - When you run "EXPLAIN (<tab>" or "EXPLAIN (S<tab>" on psql,
      "STATS" string is displayed.
=====


Finally, the attached test.sql and result.txt are a test case (incomplete)
and
its results. When the test case is completed, it will be merged into
ext_stats.sql.

Any advice is welcome!

Regards,
Tatsuro Yamada