Horton

Re: [PoC] Reducing planning time when tables have many partitions

Alvaro Herrera <alvherre@alvh.no-ip.org>

View thread
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
To: Yuya Watari <watari.yuya@gmail.com>
Cc: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, Dmitry Dolgov <9erthalion6@gmail.com>, PostgreSQL Developers <pgsql-hackers@lists.postgresql.org>, jian he <jian.universality@gmail.com>, Alena Rybakina <lena.ribackina@yandex.ru>, Andrei Lepikhov <a.lepikhov@postgrespro.ru>, David Rowley <dgrowleyml@gmail.com>, Thom Brown <thom@linux.com>, Zhang Mingli <zmlpostgres@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>
Date: 2024-12-12T12:09:26Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Update wording in optimizer/README for EquivalenceClasses

  2. Speedup child EquivalenceMember lookup in planner

  3. Allow planner to use Merge Append to efficiently implement UNION

  4. Remove trailing zero words from Bitmapsets

  5. Make Vars be outer-join-aware.

  6. Avoid making commutatively-duplicate clauses in EquivalenceClasses.

Hello Yuya,

On 2024-Dec-11, Yuya Watari wrote:

> On Tue, Dec 3, 2024 at 7:38 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> >
> > I don't think planning time in assert-enabled builds is something we
> > should worry about, at all.  Planning time in production builds is the
> > important one.
> 
> Thank you for your reply. Making debug builds too slow is not good for
> developers,

I'm repeating myself, but I disagree that this is something we should
spend _any_ time on.  Developers running assertion-enabled builds do not
care if a complicated query with one thousand partitions is planned in
500 ms instead of 300 ms.  Heck, I bet nobody cares if it took 2000 ms
either, because, you know what?  The developers don't have a thousand
partitions to begin with; if they do, it's precisely because they want
to measure this kind of effect.  This is not going to bother anyone
ever, unless you stick a hundred of these queries in the regression
tests.  In regression tests you're going to have, say, 64 partitions at
most, because having more than that doesn't test anything additional;
having that go from 40 ms to 60 ms (or whatever) isn't going to bother
anyone.

If anything, you can add a note to remove the USE_ASSERTIONS blocks once
we get past the beta process; by then any bugs will have been noticed
and the asserts will be of less value.

I would like to see this patch series get committed, and this concern
about planning time in development builds under conditions that are
unrealistic for testing is slowing the process down.  (The process is
slow enough.  This patch has already missed two releases.)  Please stop.

Memory usage and planning time in production builds is important.  You
can better spend your energy there.

Thanks

-- 
Álvaro Herrera               48°01'N 7°57'E  —  https://www.EnterpriseDB.com/
"La gente vulgar sólo piensa en pasar el tiempo;
el que tiene talento, en aprovecharlo"