Re: [PoC] Reducing planning time when tables have many partitions

Dmitry Dolgov <9erthalion6@gmail.com>

From: Dmitry Dolgov <9erthalion6@gmail.com>
To: Yuya Watari <watari.yuya@gmail.com>
Cc: PostgreSQL Developers <pgsql-hackers@lists.postgresql.org>, jian he <jian.universality@gmail.com>, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, Alena Rybakina <lena.ribackina@yandex.ru>, Andrei Lepikhov <a.lepikhov@postgrespro.ru>, David Rowley <dgrowleyml@gmail.com>, Thom Brown <thom@linux.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Zhang Mingli <zmlpostgres@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>
Date: 2024-11-27T19:51:38Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. Update wording in optimizer/README for EquivalenceClasses

  2. Speedup child EquivalenceMember lookup in planner

  3. Allow planner to use Merge Append to efficiently implement UNION

  4. Remove trailing zero words from Bitmapsets

  5. Make Vars be outer-join-aware.

  6. Avoid making commutatively-duplicate clauses in EquivalenceClasses.

> On Tue, Oct 15, 2024 at 12:20:04PM GMT, Yuya Watari wrote:
>
> The previous patches do not apply to the current master, so I have
> attached the rebased version.

Thanks for keeping it up to date.

> v25-0001
> This patch is one of the main parts of my optimization. Traditionally,
> EquivalenceClass has both parent and child members. However, this
> leads to high iteration costs when there are many child partitions. In
> v25-0001, EquivalenceClasses no longer have child members. If we need
> to iterate over child EquivalenceMembers, we use the
> EquivalenceChildMemberIterator and access the children through the
> iterator. For more details, see [1] (note that there are some design
> changes from [1]).

The referenced email containst some benchmark results. But shouldn't the
benchmark be repeated after those design changes you're talking about?

Few random notes after quickly looking through the first patch:

* There are patterns like this scattered around, it looks somewhat confusing:

	+   /* See the comments in get_eclass_for_sort_expr() to see how this works. */
	+   top_parent_rel_relids = find_relids_top_parents(root, rel->relids);

  It's not immediately clear which part of get_eclass_for_sort_expr is
  relevant, or one have to read the whole function first. Probably better to
  omit the superficial commentary on the call site, and instead expand the
  commentary for the find_relids_top_parents itself?

* The patch series features likely/unlikely since v20, but don't see any
  discussion about that. Did you notice any visible boost from that? I wonder
  how necessary that is.