Horton

Re: Decoding speculative insert with toast leaks memory

Amit Kapila <amit.kapila16@gmail.com>

View thread View on postgr.es
From: Amit Kapila <amit.kapila16@gmail.com>
To: Dilip Kumar <dilipbalaut@gmail.com>
Cc: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Andres Freund <andres@anarazel.de>, Petr Jelinek <petr.jelinek@enterprisedb.com>, Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Date: 2021-05-27T04:17:20Z
Lists: pgsql-hackers 
On Thu, May 27, 2021 at 9:40 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
>
> On Thu, May 27, 2021 at 9:26 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > >
> > > Can we consider the possibility to destroy the toast_hash in
> > > ReorderBufferCleanupTXN/ReorderBufferTruncateTXN? It will delay the
> > > clean up of memory till the end of stream or txn but there won't be
> > > any memory leak.
> > >
> >
> > The other possibility could be to clean it up when we clean the spec
> > insert change in the below code:
>
> Yeah that could be done.
>
> > /*
> > * There's a speculative insertion remaining, just clean in up, it
> > * can't have been successful, otherwise we'd gotten a confirmation
> > * record.
> > */
> > if (specinsert)
> > {
> > ReorderBufferReturnChange(rb, specinsert, true);
> > specinsert = NULL;
> > }
> >
> > But I guess we might miss cleaning it up in case of an error. A
> > similar problem could be there in the idea where we will try to tie
> > the clean up with the next change.
>
> In error case also we can handle it in the CATCH block no?
>

True, but if you do this clean-up in ReorderBufferCleanupTXN then you
don't need to take care at separate places. Also, toast_hash is stored
in txn so it appears natural to clean it up in while releasing TXN.

-- 
With Regards,
Amit Kapila.

Commits

  1. Fix ABI break introduced by commit 4daa140a2f.

  2. Fix valgrind issue in pgoutput.c.

  3. Fix decoding of speculative aborts.

  4. pgoutput: Fix memory leak due to RelationSyncEntry.map.

  5. Fix initialization of RelationSyncEntry for streaming transactions.

  6. Implement streaming mode in ReorderBuffer.