Horton

Re: [PoC] Reducing planning time when tables have many partitions

Yuya Watari <watari.yuya@gmail.com>

View thread
From: Yuya Watari <watari.yuya@gmail.com>
To: David Rowley <dgrowleyml@gmail.com>
Cc: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, Amit Langote <amitlangote09@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Dmitry Dolgov <9erthalion6@gmail.com>, PostgreSQL Developers <pgsql-hackers@lists.postgresql.org>, jian he <jian.universality@gmail.com>, Alena Rybakina <lena.ribackina@yandex.ru>, Andrei Lepikhov <a.lepikhov@postgrespro.ru>, Thom Brown <thom@linux.com>, Zhang Mingli <zmlpostgres@gmail.com>
Date: 2025-04-08T09:04:16Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Update wording in optimizer/README for EquivalenceClasses

  2. Speedup child EquivalenceMember lookup in planner

  3. Allow planner to use Merge Append to efficiently implement UNION

  4. Remove trailing zero words from Bitmapsets

  5. Make Vars be outer-join-aware.

  6. Avoid making commutatively-duplicate clauses in EquivalenceClasses.

Hello David,

On Tue, Apr 8, 2025 at 3:31 PM David Rowley <dgrowleyml@gmail.com> wrote:
>
> I've pushed the patch now. Thanks for all the reviews of my adjustments.

Thank you very much for pushing the patch! I also wish to extend my
deepest thanks to everyone who has contributed to reviewing and
improving this patch.

> Thanks to Yuya for persisting on this for so many years. I was
> impressed with that persistence and also with your very detailed and
> easy to understand performance benchmark results.  This feels like
> (all going well) it's making v18 is big win for the big partitioning
> users of Postgres. I kind of feel the "up to a few thousand partitions
> fairly well" in [1] has been abused by many or taken without the
> caveat of "the query planner to prune all but a small number of
> partitions" over the years and the "Never just assume that more
> partitions are better than fewer partitions, nor vice-versa." has been
> ignored by too many. It's good that the people pushing these limits
> will no longer be getting as big an unwelcome surprise now (I hope).
> Or, at least, onto the next bottleneck. Maybe relcache :)

This work could not have been realized without your significant
contributions. You have provided many valuable ideas and efforts since
I first proposed the patch three years ago. I would like to express my
sincere thanks for your continued and extensive support. I hope that
this work we have done together will greatly benefit users managing
highly partitioned configurations. It has truly been an honor for me
to work closely with you and all the reviewers on this work. It would
be my pleasure to continue contributing to PostgreSQL.

-- 
Best regards,
Yuya Watari