Re: queryId constant squashing does not support prepared statements
Sami Imseih <samimseih@gmail.com>
From: Sami Imseih <samimseih@gmail.com>
To: Junwang Zhao <zhjwpku@gmail.com>
Cc: Dmitry Dolgov <9erthalion6@gmail.com>,
Michael Paquier <michael@paquier.xyz>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2025-05-06T18:32:48Z
Lists: pgsql-hackers
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Fix typo in comment
- a3994ec6acb2 18.0 landed
-
Make query jumbling also squash PARAM_EXTERN params
- c2da1a5d6325 18.0 landed
-
Fix squashing algorithm for query texts
- 0f65f3eec478 18.0 landed
-
pg_stat_statements: Fix parameter number gaps in normalized queries
- 3c03b8cd7979 13.22 landed
- 8a1459f62ad1 14.19 landed
- 130300a15407 15.14 landed
- 7e8b44f4e0e6 16.10 landed
- 290e8ab32ac5 17.6 landed
- 35a428f30b15 18.0 landed
> I also agree with Alvaro that this discussion doesn't justify a
> revert. If the pre-v18 behavior wasn't chiseled on stone tablets,
> the new behavior isn't either. We can improve it some more later.
As I was looking further into what we currently have in v18 and HEAD
the normalization could break if we pass a function.
For example,
"""
select where 1 in (1, 2, int4(1));
"""
the normalized string is,
"""
select where $1 in ($2 /*, ... */))
"""
Notice the extra close parenthesis that is added after the comment. This is
because although int4(1) is a function call it is rewritten as a Const
and that breaks the assumptions being made by the location of the
last expression.
Also, something like:
"""
select where 1 in (1, 2, cast(4 as int));
"""
is normalized as:
"""
select where $1 in ($2 /*, ... */ as int))
"""
I don't think the current state is acceptable, if it results in pg_s_s
storing an invalid normalized version of the sql.
Now, with the attached v2 supporting external params, we see other normalization
anomalies such as
"""
postgres=# select where $1 in ($3, $2) and 1 in ($4, cast($5 as int))
\bind 0 1 2 3 4
postgres-# ;
--
(0 rows)
postgres=# select toplevel, query, calls from pg_stat_statements;
toplevel | query
| calls
----------+-------------------------------------------------------------------------+-------
t | select where $1 in ($2 /*, ... */) and $3 in ($4 /*, ...
*/($5 as int)) | 1
(1 row)
"""
Without properly accounting for the boundaries of the list of expressions, i.e.,
the start and end positions of '(' and ')' or '[' and ']' and normalizing the
expressions in between, it will be very difficult for the normalization to
behave sanely.
thoughts?
--
Sami Imseih
Amazon Web Services (AWS)