Re: Early WIP/PoC for inlining CTEs

Andrew Gierth <andrew@tao11.riddles.org.uk>

From: Andrew Gierth <andrew@tao11.riddles.org.uk>
To: Andreas Karlsson <andreas@proxel.se>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, Merlin Moncure <mmoncure@gmail.com>, Alvaro Herrera <alvherre@2ndquadrant.com>, Peter Eisentraut <peter.eisentraut@2ndquadrant.com>, Michael Paquier <michael@paquier.xyz>, Robert Haas <robertmhaas@gmail.com>, Andres Freund <andres@anarazel.de>, Thomas Munro <thomas.munro@enterprisedb.com>, David Fetter <david@fetter.org>, Pg Hackers <pgsql-hackers@postgresql.org>
Date: 2019-02-26T11:41:51Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. Prevent inlining of multiply-referenced CTEs with outer recursive refs.

  2. Allow user control of CTE materialization, and change the default behavior.

  3. Split QTW_EXAMINE_RTES flag into QTW_EXAMINE_RTES_BEFORE/_AFTER.

  4. document when PREPARE uses generic plans

So it turns out there's another case we have to check for, as reported
on IRC by Yaroslav Schekin (though I've simplified his query a little):

WITH RECURSIVE
  x(a) AS ((VALUES ('a'),('b'))
           UNION ALL
           (WITH
              z AS /*NOT*/ MATERIALIZED (SELECT a FROM x)
            SELECT z.a || z1.a AS a FROM z CROSS JOIN z AS z1
             WHERE length(z.a || z1.a) < 5))
SELECT * FROM x;

Here, uncommenting that NOT actually changes the result, from 22 rows to
4 rows, because we end up generating multiple worktable scans and the
recursion logic is not set up to handle that.

So what I think we need to do here is to forbid inlining if (a) the
refcount is greater than 1 and (b) the CTE in question contains,
recursively anywhere inside its rtable or the rtables of any of its
nested CTEs, a "self_reference" RTE. We only need to make this check if
we're somewhere (possibly nested) inside a recursive query, but I don't
know how practical it will be to test that condition at the point we do
inlining; doing it unconditionally will (I think) be harmless because
self_reference will not be set.

-- 
Andrew (irc:RhodiumToad)