Re: query_id: jumble names of temp tables for better pg_stat_statement UX

Sami Imseih <samimseih@gmail.com>

From: Sami Imseih <samimseih@gmail.com>
To: Lukas Fittl <lukas@fittl.com>
Cc: Michael Paquier <michael@paquier.xyz>, Tom Lane <tgl@sss.pgh.pa.us>, Christoph Berg <myon@debian.org>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, ma lz <ma100@hotmail.com>
Date: 2025-03-25T14:59:34Z
Lists: pgsql-hackers, pgsql-general
> In my experience these often not work well with pg_stat_statements today because
> of their own bloat problem, just like with temp tables. You quickly have way too many
> unique entries, and your query text file accumulates a lot of duplicative entries
> (since the same query text gets repeated in the text file, since its queryid is different),
> to the point that you can't monitor your workload at all anymore.

That is true in terms of pg_stat_statements, and I have seen users
have to increase
pg_stat_statements.max to something much higher to avoid the bloat and constant
deallocs/GC.

But, besides pg_stat_statements, queryId is used to group queries for
database load
monitoring ( pg_stat_activity sampling). As of now, different schemas
are tracked
separately, but with this change they will be merged. This may come as
a surprise to
use-cases that rely on the existing behavior.

But I do agree that pg_s_s bloat is a big pain point, so this change
should be positive
overall. Let's see if there are enough complaints to force us to reconsider.


--
Sami Imseih
Amazon Web Services (AwS)



Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. Use relation name instead of OID in query jumbling for RangeTblEntry

  2. Fix two issues with custom_query_jumble in gen_node_support.pl

  3. pg_stat_statements: Add more tests with temp tables and namespaces

  4. Add support for custom_query_jumble as a node field attribute