Re: delta relations in AFTER triggers

On Fri, May 13, 2016 at 1:02 PM, David Fetter <david@fetter.org> wrote:
> On Thu, Jan 22, 2015 at 02:41:42PM +0000, Kevin Grittner wrote:

>> [ideas on how to pass around references to ephemeral relations]
>
> [almost 17 months later]
>
> It seems like now is getting close to the time to get this into
> master.  The patch might have suffered from some bit rot, but the
> design so far seems sound.
>
> What say?

I had a talk with Tom in Brussels about this.  As I understood it,
he wasn't too keen on the suggestion by Heikki (vaguely
sorta-endorsed by Robert) of passing ephemeral named relations such
as these tuplestores around in the structures currently used for
parameter values.  He intuitively foresaw the types of problems I
had run into trying to use the same structure to pass a relation
(with structure and rows and columns of data) as is used to pass,
say, an integer.

After discussing a while, he suggested that this patch should be
looked at as an opportunity to refactor the existing handling of
the data used by AFTER triggers.  He pointed out that the existing
technique is unbounded in RAM use, with no ability to spill to
temporary files, and regularly generates complaints.  This patch is
putting all that same data into a pair of tuplestores (for old and
new row versions) that would spill to disk as needed and would
likely perform better.

I think that still leaves the question open of how best to pass
around information about ephemeral relations.  It seems to me that
threads about other features  have brushed up against similar
needs, and we should consider those in conjunction with this to
make sure we solve the problem once in a sufficiently general way
to cover all the needs.  (For example, the "asynchronous and
vectorized execution" brushes against related concepts, and I seem
to recall others.)

Unfortunately for those eager to have incremental maintenance of
materialized views (for which this patch was a milestone on the
way), Tom's suggestions seem to put additional work on that path.
When I pointed that out, he pointed out that doing features well
can take a lot of time.  I think that was about the time the next
round of drinks arrived and we decided we'd better not try to take
the discussion much further that night.  ;-)

I welcome any other thoughts on this, particularly from Tom (and
*especially* if I've misrepresented his position in any way!).
Perhaps we can get a design we all like and split the work so that
the refactoring Tom wants and the feature this patch is intended to
implement can get done in the next CF or two, and some initial
work on IMMV can still make it into the next development cycle,
building on this.

--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company