Horton

Re: unsupportable composite type partition keys

Tom Lane <tgl@sss.pgh.pa.us>

View thread View on postgr.es
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Amit Langote <amitlangote09@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2019-12-25T05:31:01Z
Lists: pgsql-hackers 
Amit Langote <amitlangote09@gmail.com> writes:
> On Tue, Dec 24, 2019 at 10:59 AM Amit Langote <amitlangote09@gmail.com> wrote:
>> Btw, does the memory leakage fix in this patch address any of the
>> pending concerns that were discussed on the "hyrax vs.
>> RelationBuildPartitionDesc" thread earlier this year[1]?
>> [1] https://www.postgresql.org/message-id/flat/3800.1560366716%40sss.pgh.pa.us#092b6b4f6bf75d2f3f90ef6a3b3eab5b

> I thought about this a little and I think it *does* address the main
> complaint in the above thread.

I experimented with the test shown in [1].  This patch does prevent that
case from accumulating copies of the partition descriptor.

(The performance of that test case is still awful, more or less O(N^2)
in the number of repetitions.  But I think what's going on is that it
repeatedly creates and deletes the same catalog entries, and we're not
smart enough to recognize that the dead row versions are fully dead,
so lots of time is wasted by unique-index checks.  It's not clear
that that's of any interest for real-world cases.)

I remain of the opinion that this is a pretty crummy, ad-hoc way to manage
the partition descriptor caching.  It's less bad than before, but I'm
still concerned that holding a relcache entry open for any long period
could result in bloat if the cache entry is rebuilt many times meanwhile
--- and there's no strong reason to think that can't happen.  Still,
maybe we can wait to solve that until there's some evidence that it
does happen in useful cases.

I also poked at the test case mentioned in the other thread about foreign
keys across lots of partitions [2].  Again, this patch gets rid of the
memory bloat, though the performance is still pretty awful with lots of
partitions for other reasons.

			regards, tom lane

[1] https://www.postgresql.org/message-id/10797.1552679128%40sss.pgh.pa.us
[2] https://www.postgresql.org/message-id/OSAPR01MB374809E8DE169C8BF2B82CBD9F6B0%40OSAPR01MB3748.jpnprd01.prod.outlook.com

Commits

  1. Fix CheckAttributeType's handling of collations for ranges.

  2. Improve comments in utils/rel.h.

  3. Allow whole-row Vars to be used in partitioning expressions.

  4. Remove equalPartitionDescs().

  5. Load relcache entries' partitioning data on-demand, not immediately.

  6. Disallow partition key expressions that return pseudo-types.

  7. Prevent a rowtype from being included in itself via a range.