Horton

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

Andrei Lepikhov <a.lepikhov@postgrespro.ru>

View thread
From: Andrei Lepikhov <a.lepikhov@postgrespro.ru>
To: Tomas Vondra <tomas.vondra@enterprisedb.com>, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, vignesh C <vignesh21@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>, Richard Guo <guofenglinux@gmail.com>, David Rowley <dgrowleyml@gmail.com>
Date: 2024-02-19T01:37:54Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add assertion to verify derived clause has constant RHS

  2. Make derived clause lookup in EquivalenceClass more efficient

  3. Doc: improve documentation for jsonpath behavior.

  4. Work around implementation restriction in adjust_appendrel_attrs.

On 19/2/2024 06:05, Tomas Vondra wrote:
> However, this is a somewhat extreme example - it's joining 5 tables,
> each with 1000 partitions, using a partition-wise join. It reduces the
> amount of memory, but the planning time is still quite high (and
> essentially the same as without the patch). So it's not like it'd make
> them significantly more practical ... do we have any other ideas/plans
> how to improve that?
The planner principle of cleaning up all allocated structures after the 
optimization stage simplifies development and code. But, if we want to 
achieve horizontal scalability on many partitions, we should introduce 
per-partition memory context and reset it in between. GEQO already has a 
short-lived memory context, making designing extensions a bit more 
challenging but nothing too painful.

-- 
regards,
Andrei Lepikhov
Postgres Professional