FW: query pg_stat_ssl hang 100%cpu

James Pang (chaolpan) <chaolpan@cisco.com>

From: "James Pang (chaolpan)" <chaolpan@cisco.com>
To: PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
Date: 2023-09-07T01:35:00Z
Lists: pgsql-bugs, pgsql-performance
Looks like an abnormal case.

From: James Pang (chaolpan)
Sent: Wednesday, September 6, 2023 9:41 AM
To: pgsql-performance@lists.postgresql.org
Subject: query pg_stat_ssl hang 100%cpu

Hi,
    PGv14.8, OS RHEL8, no SSL enabled in this database, we have a lot of client sessions who check it's ssl state by  query, all other sessions got done very quickly, but only 1 session hang there in 100% cpu tens of hours, even pg_terminate_backend does not make it stopped either.  It looks like abnormal.

   select ssl from pg_stat_ssl where pid=pg_backend_pid();

testdb=#  select pid,usename,application_name,query_start,xact_start,state_change,wait_event_type,state,query from pg_stat_activity where pid=1245344;
   pid   | usename |    application_name    |         query_start          |          xact_start           |         state_change         | wait_event_type |
state  |                         query
---------+---------+------------------------+------------------------------+-------------------------------+------------------------------+-----------------+
--------+--------------------------------------------------------
1245344 | test    | PostgreSQL JDBC Driver | 2023-09-03 02:36:23.40238+00 | 2023-09-03 02:36:23.402331+00 | 2023-09-03 02:36:23.40238+00 |                 |
active | select ssl from pg_stat_ssl where pid=pg_backend_pid()
(1 row)

testdb=#  select pid,usename,application_name,query_start,xact_start,state_change,wait_event_type,state,query from pg_stat_activity where pid=1245344;
   pid   | usename |    application_name    |         query_start          |          xact_start           |         state_change         | wait_event_type |
state  |                         query
---------+---------+------------------------+------------------------------+-------------------------------+------------------------------+-----------------+
--------+--------------------------------------------------------
1245344 | test    | PostgreSQL JDBC Driver | 2023-09-03 02:36:23.40238+00 | 2023-09-03 02:36:23.402331+00 | 2023-09-03 02:36:23.40238+00 |                 |
active | select ssl from pg_stat_ssl where pid=pg_backend_pid()
(1 row)

testdb=#  select pid,usename,application_name,query_start,xact_start,state_change,wait_event_type,state,query from pg_stat_activity where pid=1245344;
   pid   | usename |    application_name    |         query_start          |          xact_start           |         state_change         | wait_event_type |
state  |                         query
---------+---------+------------------------+------------------------------+-------------------------------+------------------------------+-----------------+
--------+--------------------------------------------------------
1245344 | test    | PostgreSQL JDBC Driver | 2023-09-03 02:36:23.40238+00 | 2023-09-03 02:36:23.402331+00 | 2023-09-03 02:36:23.40238+00 |                 |
active | select ssl from pg_stat_ssl where pid=pg_backend_pid()
(1 row)

    PID      USER        PR  NI    VIRT    RES    SHR      S  %CPU  %MEM     TIME+   nMaj nMin WCHAN         COMMAND
1245344 postgres  20   0   32.5g  12468  12164 R  99.5     0.0         4219:12    0     1343     -             postgres: test testdb 10.250.193.40(48282) BIND

#0  ensure_record_cache_typmod_slot_exists (typmod=0) at typcache.c:1714
#1  0x000000000091185b in assign_record_type_typmod (tupDesc=<optimized out>, tupDesc@entry=0x27bc738) at typcache.c:2001
#2  0x000000000091df03 in internal_get_result_type (funcid=<optimized out>, call_expr=<optimized out>, rsinfo=<optimized out>,
    resultTypeId=<optimized out>, resultTupleDesc=0x7ffc9dff8cd0) at funcapi.c:393
#3  0x000000000091e263 in get_expr_result_type (expr=expr@entry=0x2792798, resultTypeId=resultTypeId@entry=0x7ffc9dff8ccc,
    resultTupleDesc=resultTupleDesc@entry=0x7ffc9dff8cd0) at funcapi.c:230
#4  0x00000000006a2fa5 in ExecInitFunctionScan (node=node@entry=0x273afa8, estate=estate@entry=0x269e948, eflags=eflags@entry=16) at nodeFunctionscan.c:370
#5  0x000000000069084e in ExecInitNode (node=node@entry=0x273afa8, estate=estate@entry=0x269e948, eflags=eflags@entry=16) at execProcnode.c:255
#6  0x000000000068a96d in InitPlan (eflags=16, queryDesc=0x273b2d8) at execMain.c:936
#7  standard_ExecutorStart (queryDesc=0x273b2d8, eflags=16) at execMain.c:263
#8  0x00007f67c2821d5d in pgss_ExecutorStart (queryDesc=0x273b2d8, eflags=<optimized out>) at pg_stat_statements.c:965
#9  0x00000000007fc226 in PortalStart (portal=portal@entry=0x26848b8, params=params@entry=0x0, eflags=eflags@entry=0, snapshot=snapshot@entry=0x0)
    at pquery.c:514
#10 0x00000000007fa27f in exec_bind_message (input_message=0x7ffc9dff90d0) at postgres.c:1995
#11 PostgresMain (argc=argc@entry=1, argv=argv@entry=0x7ffc9dff9370, dbname=<optimized out>, username=<optimized out>) at postgres.c:4552
#12 0x000000000077a4ea in BackendRun (port=<optimized out>, port=<optimized out>) at postmaster.c:4537
#13 BackendStartup (port=<optimized out>) at postmaster.c:4259
#14 ServerLoop () at postmaster.c:1745
#15 0x000000000077b363 in PostmasterMain (argc=argc@entry=5, argv=argv@entry=0x256abc0) at postmaster.c:1417
#16 0x00000000004fec63 in main (argc=5, argv=0x256abc0) at main.c:209

Thanks,

James

Commits

  1. Fix exception safety bug in typcache.c.