Horton

Re: Removing unneeded self joins

Pogosyan Vardan <vardan.pogosyn@yandex.ru>

View thread
From: Pogosyan Vardan <vardan.pogosyn@yandex.ru>
To: Alexander Korotkov <aekorotkov@gmail.com>
Cc: Andrei Lepikhov <lepihov@gmail.com>, jian he <jian.universality@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, Robert Haas <robertmhaas@gmail.com>, Andrei Lepikhov <a.lepikhov@postgrespro.ru>, Richard Guo <guofenglinux@gmail.com>, Alexander Lakhin <exclusion@gmail.com>, "Gregory Stark (as CFM)" <stark.cfm@gmail.com>, Michał Kłeczek <michal@kleczek.org>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2024-07-18T06:26:49Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Remove GUC_NOT_IN_SAMPLE from enable_self_join_elimination

  2. Put enable_self_join_elimination into postgresql.conf.sample

  3. Get rid of ojrelid local variable in remove_rel_from_query()

  4. Implement Self-Join Elimination

  5. Revert: Remove useless self-joins

  6. Replace lateral references to removed rels in subqueries

  7. Replace relids in lateral subquery parse tree during SJE

  8. Forbid SJE with result relation

  9. Fix misuse of RelOptInfo.unique_for_rels cache by SJE

  10. Replace the relid in some missing fields during SJE

  11. Revert 56-bit relfilenode change and follow-up commits.

  12. Stabilize timetz test across DST transitions.

  13. Speed up finding EquivalenceClasses for a given set of rels

  14. Fix mark-and-restore-skipping test case to not be a self-join.

On 16.07.2024 21:30, Alexander Korotkov wrote:
> Hi, Vardan!
>
> Great, thank you!
>
> On Tue, Jul 16, 2024 at 5:26 PM Вардан Погосян<vardan.pogosyn@yandex.ru> wrote:
>> I did the SJE testing at Andrey's request.
>> To do this, I used the automatic testing tool EET (Equivalent Expression Transformation) [1] with some modifications.
>> EET transforms the logical conditions in a query, creating multiple queries waiting for the same number of rows.
> What revision of patch did you use?


I used Andrey's v4 patches.


>> In order to make sure that the SJE logic is executed at all, I tried to cover the code with ereports() as much as possible.
> Could you share this?  Probably some of these ereports() we would like to keep.

I'm not sure if it can be saved. Many reports are some messy and no 
information that is really useful for the user.


>> During the testing process, I did not find any inconsistencies in the number of rows returned, as well as other critical problems.
> Did you use assert-enabled build?  I guess you mean no server crashes,
> right?  Also, could you share some statistics on how long did you run,
> what number of queries did you execute etc.?
>
> ------
> Regards,
> Alexander Korotkov
> Supabase

Of course, i used assertion-enabled build and server didn't fail.

EET ran 10 iterations with 100 random generated queries + 100 
transformed queries => 2000 queries in total.

I can't calculate exactly how many clauses EET generates after 
transformations, but according to rough estimates for all test time, 
about 8-9% clauses detected as self join during the entire testing period.

The last query saved by EET contains 4 clauses, and after 
transformations, the new query contains ~132 clauses. SJE was not used 
in the original query, but it was used 5 times after the conversion.