Memory leak in plpython3u (with testcase and patch)
Mat Arye <mat@timescale.com>
From: Mat Arye <mat@timescale.com>
To: pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2025-01-10T18:43:00Z
Lists: pgsql-hackers
Attachments
- memory_leak_test.sql (application/octet-stream)
- memory_leak.patch (application/octet-stream) patch
Hi all, I found a memory leak in plpython3u. It happens when converting the argument for an spi plan execution (via plpy.execute() or similar). I've attached a sql file to create two plpython3u functions to reproduce the issue (memory_leak_test.sql). When running ``` set client_min_messages = 'debug'; select test_mem(1000); ``` You'll see that the memory goes to 720MB. In contrast the test_mem_query function you won't see the leak (it uses ~70MB): ``` set client_min_messages = 'debug'; select test_mem_query(1000); ``` This is because test_mem_query executes the query using a sql string instead of a spi plan with parameters. I believe the issue is in the call to `PLy_output_convert` inside `PLy_spi_execute_plan`. I've attached a patch that reduces the memory usage to ~70MB from ~720 MB when using test_mem. I based what I did on what `PLy_input_convert` does. But I am not an expert in this part of the code so I am not sure the patch is correct. In particular, I don't fully grasp the comment in `PLy_input_convert` about why the scratch is reset before not after the conversion cycle and what that has to do with python refcounts. A review by an expert would be appreciated. Thanks, Mat -- Mat Arye, Technical lead of all thing AI @ Timescale
Commits
-
Repair memory leaks in plpython.
- e98df02df3f2 17.3 landed
- bcb4db0d379f 14.16 landed
- 71bb9c4b2a14 15.11 landed
- 33a4e656dc10 16.7 landed
- 29dfffae0a6d 18.0 landed
- 02a38bc84bef 13.19 landed