Re: Lots of memory allocated when reassigning Large Objects
Tom Lane <tgl@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Guillaume Lelarge <guillaume@lelarge.info>
Cc: Justin Pryzby <pryzby@telsasoft.com>,
PostgreSQL Developers <pgsql-hackers@postgresql.org>
Date: 2021-11-29T19:39:14Z
Lists: pgsql-hackers
Attachments
- avoid-leak-in-REASSIGN-OWNED-wip.patch (text/x-diff) patch
Guillaume Lelarge <guillaume@lelarge.info> writes: > I've tried Justin's patch but it didn't help with my memory allocation > issue. FWIW, I attach the patch I used in v14. [ looks closer ... ] Ah, that patch is a bit buggy: it fails to do the right thing in the cases where the loop does a "continue". The attached revision seems to behave properly. I still see a small leakage, which I think is due to accumulation of pending sinval messages for the catalog updates. I'm curious whether that's big enough to be a problem for Guillaume's use case. (We've speculated before about bounding the memory used for pending sinval in favor of just issuing a cache reset when the list would be too big. But nobody's done anything about it, suggesting that people seldom have a problem in practice.) >> DROP OWNED BY likely has similar issues. > Didn't try it, but it wouldn't be a surprise. I tried just changing the REASSIGN to a DROP in Justin's example, and immediately hit ERROR: out of shared memory HINT: You might need to increase max_locks_per_transaction. thanks to the per-object locks we try to acquire. So I'm not sure that the DROP case can reach an interesting amount of local memory leaked before it runs out of lock-table space. regards, tom lane
Commits
-
Avoid leaking memory during large-scale REASSIGN OWNED BY operations.
- fec187dc3ca2 10.20 landed
- 8f4b0200e15a 14.2 landed
- 82d354411749 11.15 landed
- 7413caabe66e 13.6 landed
- 5cf08b4db79d 12.10 landed
- babe545caeba 15.0 landed
-
Reduce memory consumption for pending invalidation messages.
- 3aafc030a536 15.0 cited