Re: Lots of memory allocated when reassigning Large Objects
Guillaume Lelarge <guillaume@lelarge.info>
From: Guillaume Lelarge <guillaume@lelarge.info>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Justin Pryzby <pryzby@telsasoft.com>, PostgreSQL Developers <pgsql-hackers@postgresql.org>
Date: 2021-11-30T08:23:44Z
Lists: pgsql-hackers
Le mar. 30 nov. 2021 à 00:25, Tom Lane <tgl@sss.pgh.pa.us> a écrit : > Guillaume Lelarge <guillaume@lelarge.info> writes: > > Le lun. 29 nov. 2021 à 22:27, Tom Lane <tgl@sss.pgh.pa.us> a écrit : > >> I'm checking it in HEAD though; perhaps there's something else wrong > >> in the back branches? > > > That's also what I was thinking. I was only trying with v14. I just > checked > > with v15devel, and your patch works alright. So there must be something > > else with back branches. > > AFAICT the patch fixes what it intends to fix in v14 too. The reason the > residual leak is worse in v14 is that the sinval message queue is bulkier. > We improved that in HEAD in commit 3aafc030a. I wanted to make sure that commit 3aafc030a fixed this issue. So I did a few tests: without 3aafc030a, without your latest patch 1182148 kB max resident size with 3aafc030a, without your latest patch 1306812 kB max resident size without 3aafc030a, with your latest patch 1182128 kB max resident size with 3aafc030a, with your latest patch 180996 kB max resident size Definitely, 3aafc030a and your latest patch allow PostgreSQL to use much less memory. Going from 1GB to 180MB is awesome. I tried to cherry-pick 3aafc030a on v14, but it didn't apply cleanly, and the work was a bit overwhelming for me, at least in the morning. I'll try again today, but I don't have much hope. I'm not sure if I want to > take the risk of back-patching that, even now that it's aged a couple > months in the tree. It is a pretty localized fix, but it makes some > assumptions about usage patterns that might not hold up. > > I understand. Of course, it would be better if it could be fixed for each supported version but I'm already happy that it could be fixed in the next release. -- Guillaume.
Commits
-
Avoid leaking memory during large-scale REASSIGN OWNED BY operations.
- fec187dc3ca2 10.20 landed
- 8f4b0200e15a 14.2 landed
- 82d354411749 11.15 landed
- 7413caabe66e 13.6 landed
- 5cf08b4db79d 12.10 landed
- babe545caeba 15.0 landed
-
Reduce memory consumption for pending invalidation messages.
- 3aafc030a536 15.0 cited