Re: Hybrid Hash/Nested Loop joins and caching results from subplans

David Rowley <dgrowleyml@gmail.com>

From: David Rowley <dgrowleyml@gmail.com>
To: Andy Fan <zhihui.fan1213@gmail.com>
Cc: Justin Pryzby <pryzby@telsasoft.com>, Konstantin Knizhnik <k.knizhnik@postgrespro.ru>, Tom Lane <tgl@sss.pgh.pa.us>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Andres Freund <andres@anarazel.de>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2021-03-12T00:38:49Z
Lists: pgsql-hackers
On Tue, 23 Feb 2021 at 14:22, Andy Fan <zhihui.fan1213@gmail.com> wrote:
>
> On Mon, Feb 22, 2021 at 9:21 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
>>  - Maybe this should be integrated into nestloop rather than being a separate
>>    plan node.  That means that it could be dynamically enabled during
>>    execution, maybe after a few loops or after checking that there's at least
>>    some minimal number of repeated keys and cache hits.  cost_nestloop would
>>    consider whether to use a result cache or not, and explain would show the
>>    cache stats as a part of nested loop.
>
>
> +1 for this idea now.. I am always confused why there is no such node in Oracle
> even if it is so aggressive to do performance improvement and this function
> looks very promising.   After realizing the costs in planner,  I think planning time
> might be an answer (BTW, I am still not sure Oracle did this).

If you're voting for merging Result Cache with Nested Loop and making
it a single node, then that was already suggested on this thread.  I
didn't really like the idea and I wasn't alone on that. Tom didn't
much like it either. Never-the-less, I went and coded it and found
that it made the whole thing slower.

There's nothing stopping Result Cache from switching itself off if it
sees poor cache hit ratios.  It can then just become a proxy node,
effectively doing nothing apart from fetching from its own outer node
when asked for a tuple. It does not need to be part of Nested Loop to
have that ability.

David



Commits

  1. Add Result Cache executor node (take 2)

  2. Add Result Cache executor node

  3. Allow estimate_num_groups() to pass back further details about the estimation

  4. Allow users of simplehash.h to perform direct deletions

  5. Cache if PathTarget and RestrictInfos contain volatile functions

  6. Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.