Re: json(b)_array_elements use causes very large memory usage when also referencing entire json document
Tom Lane <tgl@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Andres Freund <andres@anarazel.de>
Cc: Lucas Fairchild-Madar <lucas.madar@gmail.com>, pgsql-bugs@postgresql.org
Date: 2017-10-07T00:44:20Z
Lists: pgsql-bugs
Andres Freund <andres@anarazel.de> writes: > Hm. We've a bunch of places where we free detoasted data, just out of > efficiency concerns. And since the jsonb functions are quite likely to > detoast a lot, it doesn't seem unreasonable to do so for the most likely > offenders. I mean if you've a bit more complex expression involving a > few fields accessed, freeing in the accesses will reduce maximum memory > usage by quite a bit. I'm not suggesting to work towards leak free, just > towards reducing the lifetime of a few potentially large allocations. Dunno, for the common case of not-so-large values, this would just be a net loss. pfree'ing a value we can afford to ignore till the next per-tuple context reset is not a win. Maybe we need some kind of PG_FREE_IF_LARGE_COPY macro? Where it would kick in for values over 8K or thereabouts? (You might argue that any value that got detoasted at all would be large enough to be worth worrying about; but I think that falls down because we folded short-header unshorting into the detoast mechanism.) regards, tom lane
Commits
-
Reduce memory usage of targetlist SRFs.
- 84ad4b036d97 11.0 landed
-
Fix intra-query memory leakage in nodeProjectSet.c.
- 0c25e9652461 10.1 cited