Re: Hybrid Hash/Nested Loop joins and caching results from subplans

Tom Lane <tgl@sss.pgh.pa.us>

From: Tom Lane <tgl@sss.pgh.pa.us>
To: David Rowley <dgrowleyml@gmail.com>
Cc: Andres Freund <andres@anarazel.de>, Konstantin Knizhnik <k.knizhnik@postgrespro.ru>, Andy Fan <zhihui.fan1213@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, PostgreSQL Developers <pgsql-hackers@lists.postgresql.org>
Date: 2021-03-12T01:59:48Z
Lists: pgsql-hackers
David Rowley <dgrowleyml@gmail.com> writes:
> On Tue, 23 Feb 2021 at 18:43, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> I doubt it's that bad.  We could cache such info in RestrictInfo
>> for quals, or PathTarget for tlists, without much new notational
>> overhead.  That doesn't cover everything the planner deals with
>> of course, but it would cover enough that you'd be chasing pretty
>> small returns to worry about more.

> This seems like a pretty good idea. So I coded it up.

> The 0001 patch adds a has_volatile bool field to RestrictInfo and sets
> it when building the RestrictInfo.

I'm -1 on doing it exactly that way, because you're expending
the cost of those lookups without certainty that you need the answer.
I had in mind something more like the way that we cache selectivity
estimates in RestrictInfo, in which the value is cached when first
demanded and then re-used on subsequent checks --- see in
clause_selectivity_ext, around line 750.  You do need a way for the
field to have a "not known yet" value, but that's not hard.  Moreover,
this sort of approach can be less invasive than what you did here,
because the caching behavior can be hidden inside
contain_volatile_functions, rather than having all the call sites
know about it explicitly.

			regards, tom lane



Commits

  1. Add Result Cache executor node (take 2)

  2. Add Result Cache executor node

  3. Allow estimate_num_groups() to pass back further details about the estimation

  4. Allow users of simplehash.h to perform direct deletions

  5. Cache if PathTarget and RestrictInfos contain volatile functions

  6. Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.