Horton

Re: Memory leak in WAL sender with pgoutput (v10~)

Michael Paquier <michael@paquier.xyz>

View thread View on postgr.es
From: Michael Paquier <michael@paquier.xyz>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Masahiko Sawada <sawada.mshk@gmail.com>, "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>, Euler Taveira <euler@eulerto.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Postgres hackers <pgsql-hackers@lists.postgresql.org>
Date: 2024-12-09T23:26:20Z
Lists: pgsql-hackers 
On Mon, Dec 09, 2024 at 03:36:15PM +0530, Amit Kapila wrote:
> It couldn't solve the problem completely even in back-branches. The
> SQL API case I mentioned and tested by Hou-San in the email [1] won't
> be solved.
> 
> [1] - https://www.postgresql.org/message-id/OS0PR01MB57166A4DA0ABBB94F2FBB28694362%40OS0PR01MB5716.jpnprd01.prod.outlook.com

Yeah, exactly (wanted to reply exactly that yesterday but lacked time,
thanks!).

Alvaro's solution is not perfect either as we would still bloat some
memory in the CacheMemoryContext when a single execution of the
logical decoding context finishes, but the static context approach
proposed at [2] has the merit to limit the damage on repetitive calls
when an invalidation can happen as well as in WAL senders with
periodic cleanups.  The free APIs would just lose track of these
pointers: good for WAL senders, not for repetitive pgoutput calls.

Instead of ALLOCSET_DEFAULT_SIZES, it seems to me that we should
switch to ALLOCSET_SMALL_SIZES for consistency with HEAD in Alvaro's
patch.  I would also switch "publication list context" to "logical
replication publication list context" to match with HEAD, while on it.

Would all that be OK for everybody in terms of what to do in stable
branches down to 13?

[2]: https://www.postgresql.org/message-id/202411300828.hwe55pzx5a4x@alvherre.pgsql
--
Michael

Commits

  1. Fix memory leak in pgoutput with relation attribute map

  2. Fix memory leak in pgoutput with publication list cache

  3. Allow specifying row filters for logical replication of tables.