Horton

reorganizing partitioning code (was: Re: [HACKERS] path toward faster partition pruning)

Amit Langote <langote_amit_f8@lab.ntt.co.jp>

View thread View on postgr.es
From: Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>, Robert Haas <robertmhaas@gmail.com>
Cc: Ashutosh Bapat <ashutosh.bapat@enterprisedb.com>, Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>, Jesper Pedersen <jesper.pedersen@redhat.com>, David Rowley <david.rowley@2ndquadrant.com>, Amit Langote <amitlangote09@gmail.com>, Rajkumar Raghuwanshi <rajkumar.raghuwanshi@enterprisedb.com>, Dilip Kumar <dilipbalaut@gmail.com>, Beena Emerson <memissemerson@gmail.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2018-02-13T08:47:43Z
Lists: pgsql-hackers

Attachments

On 2018/02/09 2:58, Alvaro Herrera wrote:
> Robert Haas wrote:
>> On Wed, Feb 7, 2018 at 3:42 AM, Ashutosh Bapat
>> <ashutosh.bapat@enterprisedb.com> wrote:
> 
>>> partition.c seems to have two kinds of functions 1. that build and
>>> manage relcache, creates quals from bounds etc. which are metadata
>>> management kind 2. partition bound comparison functions, and other
>>> optimizer related functions. May be we should divide the file that
>>> way. The first category code remains in catalog/ as it is today. The
>>> second catagory functions move to optimizer/.
>>
>> It would be sensible to separate functions that build and manage data
>> in the relcache from other functions.  I think we should consider
>> moving the existing functions of that type from partition.c to
>> src/backend/utils/cache/partcache.c.
> 
> FWIW I've been thinking that perhaps we need some other separation of
> code better than statu quo.  The current partition.c file includes stuff
> for several modules and ISTM all these new patches are making more and
> more of a mess.  So +1 to the general idea of splitting things up.
> Maybe partcache.c is not ambitious enough, but it seems a good first
> step.

Agree with the proposed reorganizing and adding a partcache.c, which I
tried to do in the attached patch.

* The new src/backend/utils/cache/partcache.c contains functions that
initialize relcache's partitioning related fields.  Various partition
bound comparison and search functions (and then some) that work off of the
cached information are moved.  Also, since we cache partition qual,
interface functions RelationGetPartitioQual(Relation) and
get_partition_qual_relid(Oid) are moved too.

* The new src/include/utils/partcache.h contains various struct
definitions that are moved from backend/catalog/partition.c,
include/catalog/partition.h, and include/utils/rel.h.  Also, declarations
of interface functions of partcache.c.

Thoughts?

Thanks,
Amit

Commits

  1. Fix assorted partition pruning bugs

  2. Make gen_partprune_steps static

  3. Remove useless 'default' clause

  4. Reorganize partitioning code

  5. Use custom hash opclass for hash partition pruning

  6. Blindly attempt to fix sepgsql tests broken due to 9fdb675fc5.

  7. Attempt to fix endianess issues in new hash partition test.

  8. Faster partition pruning

  9. For partitionwise join, match on partcollation, not parttypcoll.

  10. Revise API for partition bound search functions.

  11. Revise API for partition_rbound_cmp/partition_rbound_datum_cmp.

  12. Fix possible crash in partition-wise join.

  13. Refactor code for partition bound searching

  14. New C function: bms_add_range

  15. Add extensive tests for partition pruning.

  16. Add null test to partition constraint for default range partitions.

  17. Remove BufFile's isTemp flag.

  18. Make OWNER TO subcommand mention consistent

  19. Fix index matching for operators with mixed collatable/noncollatable inputs.