Horton

Thread

Commits

  1. Fix two memory leaks around force-storing tuples in slots.

  1. ExecForceStoreMinimalTuple leaks memory like there's no tomorrow

    Tom Lane <tgl@sss.pgh.pa.us> — 2019-04-16T02:46:56Z

    Using HEAD,

create table t1 as select generate_series(1,40000000) id;
vacuum analyze t1;
explain select * from t1, t1 t1b where t1.id = t1b.id;
-- should indicate a hash join
explain analyze select * from t1, t1 t1b where t1.id = t1b.id;

... watch the process's memory consumption bloat.  (It runs for
awhile before that starts to happen, but eventually it goes to
a couple of GB.)

It looks to me like the problem is that ExecHashJoinGetSavedTuple
calls ExecForceStoreMinimalTuple with shouldFree = true, and
ExecForceStoreMinimalTuple's second code branch simply ignores
the requirement to free the supplied tuple.

			regards, tom lane

  2. Re: ExecForceStoreMinimalTuple leaks memory like there's no tomorrow

    Michael Paquier <michael@paquier.xyz> — 2019-04-16T03:31:37Z

    On Mon, Apr 15, 2019 at 10:46:56PM -0400, Tom Lane wrote:
> create table t1 as select generate_series(1,40000000) id;
> vacuum analyze t1;
> explain select * from t1, t1 t1b where t1.id = t1b.id;
> -- should indicate a hash join
> explain analyze select * from t1, t1 t1b where t1.id = t1b.id;
> 
> ... watch the process's memory consumption bloat.  (It runs for
> awhile before that starts to happen, but eventually it goes to
> a couple of GB.)
> 
> It looks to me like the problem is that ExecHashJoinGetSavedTuple
> calls ExecForceStoreMinimalTuple with shouldFree = true, and
> ExecForceStoreMinimalTuple's second code branch simply ignores
> the requirement to free the supplied tuple.

Open item added, as the root comes from 4da597ed.
--
Michael

  3. Re: ExecForceStoreMinimalTuple leaks memory like there's no tomorrow

    Andres Freund <andres@anarazel.de> — 2019-04-19T02:04:09Z

    Hi,

On 2019-04-15 22:46:56 -0400, Tom Lane wrote:
> Using HEAD,
> 
> create table t1 as select generate_series(1,40000000) id;
> vacuum analyze t1;
> explain select * from t1, t1 t1b where t1.id = t1b.id;
> -- should indicate a hash join
> explain analyze select * from t1, t1 t1b where t1.id = t1b.id;
> 
> ... watch the process's memory consumption bloat.  (It runs for
> awhile before that starts to happen, but eventually it goes to
> a couple of GB.)
> 
> It looks to me like the problem is that ExecHashJoinGetSavedTuple
> calls ExecForceStoreMinimalTuple with shouldFree = true, and
> ExecForceStoreMinimalTuple's second code branch simply ignores
> the requirement to free the supplied tuple.

Thanks for finding. The fix is obviously easy - but looking through the
code I think I found another similar issue. I'll fix both in one go
tomorrow.

Greetings,

Andres Freund

  4. Re: ExecForceStoreMinimalTuple leaks memory like there's no tomorrow

    Andres Freund <andres@anarazel.de> — 2019-04-19T18:55:45Z

    Hi,

On 2019-04-18 19:04:09 -0700, Andres Freund wrote:
> On 2019-04-15 22:46:56 -0400, Tom Lane wrote:
> > Using HEAD,
> > 
> > create table t1 as select generate_series(1,40000000) id;
> > vacuum analyze t1;
> > explain select * from t1, t1 t1b where t1.id = t1b.id;
> > -- should indicate a hash join
> > explain analyze select * from t1, t1 t1b where t1.id = t1b.id;
> > 
> > ... watch the process's memory consumption bloat.  (It runs for
> > awhile before that starts to happen, but eventually it goes to
> > a couple of GB.)
> > 
> > It looks to me like the problem is that ExecHashJoinGetSavedTuple
> > calls ExecForceStoreMinimalTuple with shouldFree = true, and
> > ExecForceStoreMinimalTuple's second code branch simply ignores
> > the requirement to free the supplied tuple.
> 
> Thanks for finding. The fix is obviously easy - but looking through the
> code I think I found another similar issue. I'll fix both in one go
> tomorrow.

Pushed the combined fix for that. Thanks!

Greetings,

Andres Freund