Horton

Re: [PoC] Reducing planning time when tables have many partitions

Andrei Lepikhov <a.lepikhov@postgrespro.ru>

View thread
From: Andrey Lepikhov <a.lepikhov@postgrespro.ru>
To: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Cc: Yuya Watari <watari.yuya@gmail.com>, Thom Brown <thom@linux.com>, David Rowley <dgrowleyml@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Zhang Mingli <zmlpostgres@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, PostgreSQL Developers <pgsql-hackers@lists.postgresql.org>
Date: 2023-08-08T03:22:49Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Update wording in optimizer/README for EquivalenceClasses

  2. Speedup child EquivalenceMember lookup in planner

  3. Allow planner to use Merge Append to efficiently implement UNION

  4. Remove trailing zero words from Bitmapsets

  5. Make Vars be outer-join-aware.

  6. Avoid making commutatively-duplicate clauses in EquivalenceClasses.

On 7/8/2023 19:15, Ashutosh Bapat wrote:
> 
> 
> On Mon, Aug 7, 2023 at 2:21 PM Andrey Lepikhov 
> <a.lepikhov@postgrespro.ru <mailto:a.lepikhov@postgrespro.ru>> wrote:
> 
>      >> Do you think that the memory measurement patch I have shared in
>     those threads is useful in itself? If so, I will start another
>     proposal to address it.
>      >
>      > For me, who is developing the planner in this thread, the memory
>      > measurement patch is useful. However, most users do not care about
>      > memory usage, so there is room for consideration. For example, making
>      > the metrics optional in EXPLAIN ANALYZE outputs might be better.
>      >
>     +1. Any memory-related info in the output of EXPLAIN ANALYZE makes
>     tests
>     more complex because of architecture dependency.
> 
> 
> As far as the tests go, the same is the case with planning time and 
> execution time. They change even without changing the architecture. But 
> we have tests which mask the actual values. Something similar will be 
> done to the planning memory.
It is a positive thing to access some planner internals from the 
console, of course. My point is dedicated to the structuration of an 
EXPLAIN output and is caused by two reasons:
1. I use the EXPLAIN command daily to identify performance issues and 
the optimiser's weak points. According to the experience, when you have 
an 'explain analyze' containing more than 100 strings, you try removing 
unnecessary information to improve observability. It would be better to 
have the possibility to see an EXPLAIN with different levels of the 
output details. Flexibility here reduces a lot of manual work, sometimes.
2. Writing extensions and having an explain analyze in the regression 
test, we must create masking functions just to make the test more 
stable. That additional work can be avoided with another option, like 
MEMUSAGE ON/OFF.

So, in my opinion, it would be better to introduce this new output data 
guarded by additional option.

> 
> I will propose it as a separate patch in the next commitfest and will 
> seek opinions from other hackers.
Cool, good news.

-- 
regards,
Andrey Lepikhov
Postgres Professional