Re: Lots of memory allocated when reassigning Large Objects
Justin Pryzby <pryzby@telsasoft.com>
From: Justin Pryzby <pryzby@telsasoft.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Guillaume Lelarge <guillaume@lelarge.info>, pgsql-hackers@postgresql.org
Date: 2021-11-30T00:55:22Z
Lists: pgsql-hackers
On Mon, Nov 29, 2021 at 01:40:31PM -0500, Tom Lane wrote:
> DROP OWNED BY likely has similar issues.
I tried a few more commands but found no significant issue.
IMO if you have 100k tables, then you can afford 1GB RAM.
SELECT format('CREATE TABLE t%s()', a) FROM generate_series(1,9999)a\gexec
SET client_min_messages=debug; SET log_statement_stats=on;
CREATE TABLESPACE tbsp LOCATION '/home/pryzbyj/tblspc';
ALTER TABLE ALL IN TABLESPACE pg_default SET TABLESPACE tbsp;
-- 10k tables uses 78MB RAM, which seems good enough (64MB the 2nd time??)
GRANT ALL ON ALL TABLES IN SCHEMA public TO current_user;
-- 10k tables uses 50MB RAM, which seems good enough
GRANT ALL ON ALL SEQUENCES IN SCHEMA public TO current_user
-- 10k sequences uses 47MB RAM, which seems good enough
SELECT format('CREATE FUNCTION f%s() RETURNS int RETURN 1;', a) FROM generate_series(1,9999)a;
GRANT ALL ON ALL FUNCTIONS IN SCHEMA public TO current_user;
-- 10k functions uses 62MB RAM, which seems good enough
And it looks like for ALTER PUBLICATION .. FOR ALL TABLES IN SCHEMA, the
namespace itself is stored, rather than enumerating all its tables.
>> IOW, it's asking for at least 481MB to reassign 1 million empty LO. It
>> strikes me as odd.
@Guillaume: Even if memory use with the patch isn't constant, I imagine it's
enough to have avoided OOM.
--
Justin
Commits
-
Avoid leaking memory during large-scale REASSIGN OWNED BY operations.
- fec187dc3ca2 10.20 landed
- 8f4b0200e15a 14.2 landed
- 82d354411749 11.15 landed
- 7413caabe66e 13.6 landed
- 5cf08b4db79d 12.10 landed
- babe545caeba 15.0 landed
-
Reduce memory consumption for pending invalidation messages.
- 3aafc030a536 15.0 cited