Horton

Re: Memory leak in WAL sender with pgoutput (v10~)

vignesh C <vignesh21@gmail.com>

View thread View on postgr.es
From: vignesh C <vignesh21@gmail.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>, "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>, Euler Taveira <euler@eulerto.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Postgres hackers <pgsql-hackers@lists.postgresql.org>
Date: 2024-12-10T03:24:19Z
Lists: pgsql-hackers

Attachments

On Tue, 10 Dec 2024 at 04:56, Michael Paquier <michael@paquier.xyz> wrote:
>
> On Mon, Dec 09, 2024 at 03:36:15PM +0530, Amit Kapila wrote:
> > It couldn't solve the problem completely even in back-branches. The
> > SQL API case I mentioned and tested by Hou-San in the email [1] won't
> > be solved.
> >
> > [1] - https://www.postgresql.org/message-id/OS0PR01MB57166A4DA0ABBB94F2FBB28694362%40OS0PR01MB5716.jpnprd01.prod.outlook.com
>
> Yeah, exactly (wanted to reply exactly that yesterday but lacked time,
> thanks!).
>
> Alvaro's solution is not perfect either as we would still bloat some
> memory in the CacheMemoryContext when a single execution of the
> logical decoding context finishes, but the static context approach
> proposed at [2] has the merit to limit the damage on repetitive calls
> when an invalidation can happen as well as in WAL senders with
> periodic cleanups.  The free APIs would just lose track of these
> pointers: good for WAL senders, not for repetitive pgoutput calls.
>
> Instead of ALLOCSET_DEFAULT_SIZES, it seems to me that we should
> switch to ALLOCSET_SMALL_SIZES for consistency with HEAD in Alvaro's
> patch.  I would also switch "publication list context" to "logical
> replication publication list context" to match with HEAD, while on it.

Yes, that makes sense. How about something like the attached patch.

Regards,
Vignesh

Commits

  1. Fix memory leak in pgoutput with relation attribute map

  2. Fix memory leak in pgoutput with publication list cache

  3. Allow specifying row filters for logical replication of tables.