Re: [PoC] Reducing planning time when tables have many partitions

Yuya Watari <watari.yuya@gmail.com>

From: Yuya Watari <watari.yuya@gmail.com>

To: David Rowley <dgrowleyml@gmail.com>

Cc: Thom Brown <thom@linux.com>, Andrey Lepikhov <a.lepikhov@postgrespro.ru>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Zhang Mingli <zmlpostgres@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, PostgreSQL Developers <pgsql-hackers@lists.postgresql.org>

Date: 2023-01-27T03:48:30Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Update wording in optimizer/README for EquivalenceClasses
- d7c04db27aeb 18.0 landed
Speedup child EquivalenceMember lookup in planner
- d69d45a5a956 18.0 landed
Allow planner to use Merge Append to efficiently implement UNION
- 66c0185a3d14 17.0 cited
Remove trailing zero words from Bitmapsets
- a8c09daa8bb1 17.0 cited
Make Vars be outer-join-aware.
- 2489d76c4906 16.0 cited
Avoid making commutatively-duplicate clauses in EquivalenceClasses.
- a5fc46414deb 16.0 cited

Attachments

v12-0001-Add-Bitmapset-indexes-for-faster-lookup-of-Equiv.patch (application/octet-stream) patch v12-0001
v12-0002-Adjust-bms_int_members-so-that-it-shortens-the-l.patch (application/octet-stream) patch v12-0002
v12-0003-Add-iterators-for-looping-over-EquivalenceMember.patch (application/octet-stream) patch v12-0003
v12-0004-Add-a-validation-to-check-that-two-iteration-res.patch (application/octet-stream) patch v12-0004

Dear David,

On Mon, Dec 12, 2022 at 1:50 PM David Rowley <dgrowleyml@gmail.com> wrote:
> I've attached the v11 set of patches.

Thanks for creating the v11 version. I think your patches look good to
me. I really apologize for my late reply.

> a) I think the iterator code should have some additional sanity checks
> that the results of both methods match when building with
> USE_ASSERT_CHECKING. I've got some concerns that we might break
> something. The logic about what the em_relids is set to for child
> members is a little confusing. See add_eq_member().

I added sanity checking code to check that two iteration results are
the same. I have attached a new version of the patch, v12, to this
email.

The implementation of my sanity checking code (v12-0004) is not ideal
and a little ugly. I understand that and will try to improve it.

However, there is more bad news. Unfortunately, some regression tests
are failing in my environment. I'm not sure why, but it could be that
a) my sanity checking code (v12-0004) is wrong, or b) our patches have
some bugs.

I will investigate this issue further, and share the results when found.

-- 
Best regards,
Yuya Watari