Re: Hybrid Hash/Nested Loop joins and caching results from subplans

Andres Freund <andres@anarazel.de>

From: Andres Freund <andres@anarazel.de>
To: David Rowley <dgrowleyml@gmail.com>
Cc: Konstantin Knizhnik <k.knizhnik@postgrespro.ru>, Andy Fan <zhihui.fan1213@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, Alvaro Herrera <alvherre@alvh.no-ip.org>, PostgreSQL Developers <pgsql-hackers@lists.postgresql.org>
Date: 2021-02-23T01:39:11Z
Lists: pgsql-hackers
Hi,

On 2021-02-16 23:15:51 +1300, David Rowley wrote:
> There might be gains to be had by checking the parse once rather than
> having to call contains_volatile_functions in the various places we do
> call it. I think both the parallel safety and volatile checks could
> then be done in the same tree traverse. Anyway. I've not done any
> hacking on this.  It's just an idea so far.

ISTM that it could be worth to that as part of preprocess_expression() -
it's a pass that we unconditionally do pretty early, it already computes
opfuncid, often already fetches the pg_proc entry (cf
simplify_function()), etc.

Except for the annoying issue that that we pervasively use Lists as
expressions, I'd argue that we should actually cache "subtree
volatility" in Expr nodes, similar to the way we use OpExpr.opfuncid
etc. That'd allow us to make contain_volatile_functions() very cheap in
the majority of cases, but we could still easily invalidate that state
when necessary by setting "exprhasvolatile" to unknown (causing the next
contain_volatile_functions() to compute it from scratch).

But since we actually do use Lists as expressions (which do not inherit
from Expr), we'd instead need to pass a new param to
preprocess_expression() that stores the volatility somewhere in
PlannerInfo? Seems a bit yucky to manage :(.

Greetings,

Andres Freund



Commits

  1. Add Result Cache executor node (take 2)

  2. Add Result Cache executor node

  3. Allow estimate_num_groups() to pass back further details about the estimation

  4. Allow users of simplehash.h to perform direct deletions

  5. Cache if PathTarget and RestrictInfos contain volatile functions

  6. Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.