Horton

Re: Decoding speculative insert with toast leaks memory

Peter Geoghegan <pg@bowt.ie>

View thread View on postgr.es
From: Peter Geoghegan <pg@bowt.ie>
To: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Andres Freund <andres@anarazel.de>, Petr Jelinek <petr.jelinek@enterprisedb.com>, Peter Eisentraut <peter.eisentraut@2ndquadrant.com>, Amit Kapila <amit.kapila16@gmail.com>
Date: 2021-05-27T02:56:51Z
Lists: pgsql-hackers 
On Wed, Mar 24, 2021 at 10:34 PM Ashutosh Bapat
<ashutosh.bapat.oss@gmail.com> wrote:
> Hi All,
> We saw OOM in a system where WAL sender consumed Gigabttes of memory
> which was never released. Upon investigation, we found out that there
> were many ReorderBufferToastHash memory contexts linked to
> ReorderBuffer context, together consuming gigs of memory. They were
> running INSERT ... ON CONFLICT .. among other things. A similar report
> at [1]

What is the relationship between this bug and commit 7259736a6e5,
dealt specifically with TOAST and speculative insertion resource
management issues within reorderbuffer.c? Amit?

-- 
Peter Geoghegan

Commits

  1. Fix ABI break introduced by commit 4daa140a2f.

  2. Fix valgrind issue in pgoutput.c.

  3. Fix decoding of speculative aborts.

  4. pgoutput: Fix memory leak due to RelationSyncEntry.map.

  5. Fix initialization of RelationSyncEntry for streaming transactions.

  6. Implement streaming mode in ReorderBuffer.