Re: Get memory contexts of an arbitrary backend process

On 2021-03-05 14:22, Fujii Masao wrote:
> On 2021/03/04 18:32, torikoshia wrote:
>> On 2021-01-14 19:11, torikoshia wrote:
>>> Since pg_get_target_backend_memory_contexts() waits to dump memory 
>>> and
>>> it could lead dead lock as below.
>>> 
>>>   - session1
>>>   BEGIN; TRUNCATE t;
>>> 
>>>   - session2
>>>   BEGIN; TRUNCATE t; -- wait
>>> 
>>>   - session1
>>>   SELECT * FROM pg_get_target_backend_memory_contexts(<pid of session
>>> 2>); --wait
>>> 
>>> 
>>> Thanks for notifying me, Fujii-san.
>>> 
>>> 
>>> Attached v8 patch that prohibited calling the function inside 
>>> transactions.
>> 
>> Regrettably, this modification could not cope with the advisory lock 
>> and
>> I haven't come up with a good way to deal with it.
>> 
>> It seems to me that the architecture of the requestor waiting for the
>> dumper leads to this problem and complicates things.
>> 
>> 
>> Considering the discussion printing backtrace discussion[1], it seems
>> reasonable that the requestor just sends a signal and dumper dumps to
>> the log file.
> 
> +1

Thanks!

I remade the patch and introduced a function
pg_print_backend_memory_contexts(PID) which prints the memory contexts 
of
the specified PID to elog.

   =# SELECT pg_print_backend_memory_contexts(450855);

   ** log output **
   2021-03-17 15:21:01.942 JST [450855] LOG:  Printing memory contexts of 
PID 450855
   2021-03-17 15:21:01.942 JST [450855] LOG:  level: 0 TopMemoryContext: 
68720 total in 5 blocks; 16312 free (15 chunks); 52408 used
   2021-03-17 15:21:01.942 JST [450855] LOG:  level: 1 Prepared Queries: 
65536 total in 4 blocks; 35088 free (14 chunks); 30448 used
   2021-03-17 15:21:01.942 JST [450855] LOG:  level: 1 pgstat 
TabStatusArray lookup hash table: 8192 total in 1 blocks; 1408 free (0 
chunks); 6784 used
   ..(snip)..
   2021-03-17 15:21:01.942 JST [450855] LOG:  level: 2 CachedPlanSource: 
4096 total in 3 blocks; 680 free (0 chunks); 3416 used: PREPARE hoge_200 
AS SELECT * FROM pgbench_accounts WHERE aid = 
1111111111111111111111111111111111111...
   2021-03-17 15:21:01.942 JST [450855] LOG:  level: 3 CachedPlanQuery: 
4096 total in 3 blocks; 464 free (0 chunks); 3632 used
   ..(snip)..
   2021-03-17 15:21:01.945 JST [450855] LOG:  level: 1 Timezones: 104128 
total in 2 blocks; 2584 free (0 chunks); 101544 used
   2021-03-17 15:21:01.945 JST [450855] LOG:  level: 1 ErrorContext: 8192 
total in 1 blocks; 7928 free (5 chunks); 264 used
   2021-03-17 15:21:01.945 JST [450855] LOG:  Grand total: 2802080 bytes 
in 1399 blocks; 480568 free (178 chunks); 2321512 used


As above, the output is almost the same as MemoryContextStatsPrint()
except for the way of expression of the level.
MemoryContextStatsPrint() uses indents, but
pg_print_backend_memory_contexts() writes it as "level: %d".

Since there was discussion about enlarging StringInfo may cause
errors on OOM[1], this patch calls elog for each context.

As with MemoryContextStatsPrint(), each context shows 100
children at most.
I once thought it should be configurable, but something like
pg_print_backend_memory_contexts(PID, num_children) needs to send
the 'num_children' from requestor to dumper and it seems to require
another infrastructure.
Creating a new GUC for this seems overkill.
If MemoryContextStatsPrint(), i.e. showing 100 children at most is
enough, this hard limit may be acceptable.

Only superusers can call pg_print_backend_memory_contexts().

I'm going to add documentation and regression tests.


Any thoughts?


[1] 
https://www.postgresql.org/message-id/CAMsr%2BYGh%2Bsso5N6Q%2BFmYHLWC%3DBPCzA%2B5GbhYZSGruj2d0c7Vvg%40mail.gmail.com
"r_d/strengthen_perf/print_memcon.md" 110L, 5642C written


Regards,

--
Atsushi Torikoshi
NTT DATA CORPORATION