Re: queryId constant squashing does not support prepared statements
Sami Imseih <samimseih@gmail.com>
From: Sami Imseih <samimseih@gmail.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Álvaro Herrera <alvherre@kurilemu.de>, Dmitry Dolgov <9erthalion6@gmail.com>, Junwang Zhao <zhjwpku@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2025-05-28T21:05:03Z
Lists: pgsql-hackers
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Fix typo in comment
- a3994ec6acb2 18.0 landed
-
Make query jumbling also squash PARAM_EXTERN params
- c2da1a5d6325 18.0 landed
-
Fix squashing algorithm for query texts
- 0f65f3eec478 18.0 landed
-
pg_stat_statements: Fix parameter number gaps in normalized queries
- 3c03b8cd7979 13.22 landed
- 8a1459f62ad1 14.19 landed
- 130300a15407 15.14 landed
- 7e8b44f4e0e6 16.10 landed
- 290e8ab32ac5 17.6 landed
- 35a428f30b15 18.0 landed
Attachments
- v7-0003-Fix-Normalization-for-squashed-query-texts.patch (application/octet-stream) patch v7-0003
- v7-0001-Fix-broken-normalization-due-to-duplicate-constan.patch (application/octet-stream) patch v7-0001
- v7-0002-Enhanced-query-jumbling-squashing-tests.patch (application/octet-stream) patch v7-0002
- v7-0004-Support-Squashing-of-External-Parameters.patch (application/octet-stream) patch v7-0004
>> * 0001:
> You have mentioned the addition of tests, but v6-0001 includes nothing
> of the kind. Am I missing something? How much coverage did you
> intend to add here? These seem to be included in squashing.sql in
> patch v6-0002, but IMO this should be moved somewhere else to work
> with the back-branches and make the whole backpatch story more
> consistent.
That's my mistake. I added a new file called normalize.sql to test
specific normalization scenarios. Added in v7
> > * 0002:
> RelabelType perhaps?
Fixed.
> A lot of the tests introduced in v6-0002 are copy-pastes of the
> previous ones for IN clauses introduced for the ARRAY cases, with
> comments explaining the reasons why lists are squashed or not also
> copy-pasted. Perhaps it would make sense to group the ARRAY and IN
> clause cases together. For example, group each of the two CoerceViaIO
> cases together in a single query on pg_stat_statements, with a single
> pg_stat_statements_reset(). That would make more difficult to miss
> the fact that we need to care about IN clauses *and* arrays when
> adding more test patterns, if we add some of course.
>
I agree. I reorganized by grouping both for IN and ARRAY tests
together for a specific test area.
I also clarified some comments in the tests, etc.
> > * 0003:
> I'd suggest to not use "n" for this one, but a different variable
> name, leaving the internals for the SubLink cases minimally touched.
I agree. Fixed.
> Implementation-wise, I would choose a location with a query length
> rather than start and end locations. That's what we do for the nested
> queries in the DMLs, so on consistency grounds..
This is different because the existing location field is tracking
something a bit different than what we want to track.
What the current location field is tracking is to assist in things
like error messages, like below, which wants to place the
caret (^) in the proper location, which is at the location of the
"IN".
```
ERROR: operator does not exist: oid = text
LINE 1: select where 1::oid IN (1::text, 2, 3);
^
HINT: No operator matches the given name and argument types. You
might need to add explicit type casts.
test=#
```
What we need for squashing is to track the start of the outer '(' and ')' of
the expression.
I could do something like fields to track list_start and list_length instead,
Will that be better to be closer in consistency?
> > * 0004: implements external parameter squashing.
> Using a custom implementation for Param nodes means that we are going
> to apply a location record for all external parameters, not only the
> ones in the lists.. Not sure if this is a good idea. Something
> smells a bit wrong with this approach. Sorry, I cannot push my finger
> on what exactly when typing this paragraph.
Actually, only the parameters outside of the squashed lists are
recorded. I added
a comment to make that clear. I would really want to only record parameter
locations if we know we have a squashed list, but it's impossible to
know that in
advance.
Also, the reason for a custom implementation for Param is to avoid having
to change the signature of JUMBLE_LOCATION because we have a
new bool argument to RecordExpressionLocation to set a location as an
external parameter. We will also need special handling in gen_node_support.pl
for Param to set the new argument. I was not too happy with doing that.
> Patches 0002 and 0003 fix bugs in the squashing logic present only on
> HEAD, nothing that impacts older branches already released, right?
That is correct.
--
Sami