Horton

Re: [PoC] Reducing planning time when tables have many partitions

Yuya Watari <watari.yuya@gmail.com>

View thread
From: Yuya Watari <watari.yuya@gmail.com>
To: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Cc: David Rowley <dgrowleyml@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Dmitry Dolgov <9erthalion6@gmail.com>, PostgreSQL Developers <pgsql-hackers@lists.postgresql.org>, jian he <jian.universality@gmail.com>, Alena Rybakina <lena.ribackina@yandex.ru>, Andrei Lepikhov <a.lepikhov@postgrespro.ru>, Thom Brown <thom@linux.com>, Zhang Mingli <zmlpostgres@gmail.com>
Date: 2025-03-31T05:47:16Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Update wording in optimizer/README for EquivalenceClasses

  2. Speedup child EquivalenceMember lookup in planner

  3. Allow planner to use Merge Append to efficiently implement UNION

  4. Remove trailing zero words from Bitmapsets

  5. Make Vars be outer-join-aware.

  6. Avoid making commutatively-duplicate clauses in EquivalenceClasses.

Hello Ashutosh,

Thank you for your detailed review, and apologies for my delayed response.

On Thu, Mar 27, 2025 at 1:42 PM Ashutosh Bapat
<ashutosh.bapat.oss@gmail.com> wrote:
>
> Here are memory consumption numbers using list_free() instead of pfree(), using the same method as [1], using a binary without asserts and debug info. PFA the patchset where all the patches are the same as v35 but with an extra patch fixing memory leak. The memory leak is visible with a higher number of joins. At a lower number of joins, I expect that the memory saved is less than a KB or the leaked memory fits within 1 chunk of memory context and hence not visible.

Thank you for conducting your benchmarks. Your results clearly show
increased memory consumption with my patches. As Tom also suggested,
we may reduce memory usage by adopting a different design. I will
reconsider alternative approaches and compare the memory usage to the
current version.

Thank you once again for conducting the benchmarks.

-- 
Best regards,
Yuya Watari