Horton

Re: [HACKERS] Runtime Partition Pruning

Amit Langote <langote_amit_f8@lab.ntt.co.jp>

View thread View on postgr.es
From: Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>
To: Robert Haas <robertmhaas@gmail.com>, David Rowley <david.rowley@2ndquadrant.com>
Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>, Amit Langote <amitlangote09@gmail.com>, Jesper Pedersen <jesper.pedersen@redhat.com>, Rajkumar Raghuwanshi <rajkumar.raghuwanshi@enterprisedb.com>, Beena Emerson <memissemerson@gmail.com>, amul sul <sulamul@gmail.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>, Dilip Kumar <dilipbalaut@gmail.com>
Date: 2018-04-13T05:38:46Z
Lists: pgsql-hackers

Attachments

On 2018/04/13 1:57, Robert Haas wrote:
>> It might be possible to do something better in each module by keeping
>> an array indexed by RTI which have each entry NULL initially then on
>> first relation_open set the element in the array to that pointer.
> 
> I'm not sure that makes a lot of sense in the planner, but in the
> executor it might be a good idea.   See also
> https://www.postgresql.org/message-id/CA%2BTgmoYKToP4-adCFFRNrO21OGuH%3Dphx-fiB1dYoqksNYX6YHQ%40mail.gmail.com
> for related discussion.  I think that a coding pattern where we rely
> on relation_open(..., NoLock) is inherently dangerous -- it's too easy
> to be wrong about whether the lock is sure to have been taken.  It
> would be much better to open the relation once and hold onto the
> pointer, not just for performance reasons, but for robustness.

About the specific relation_open(.., NoLock) under question, I think there
might be a way to address this by opening the tables with the appropriate
lock mode in partitioned_rels list in ExecLockNonleafAppendTables and
keeping the Relation pointers around until ExecEndNode.  Then instead of
ExecSetupPartitionPruneState doing relation_open/close(.., NoLock), it
just reuses the one that's passed by the caller.

Attached a PoC patch.  David, thoughts?

Thanks,
Amit

Commits

  1. Initialize ExprStates once in run-time partition pruning

  2. Add bms_prev_member function

  3. Support partition pruning at execution time

  4. Document partprune.c a little better

  5. Fix crash in pg_replication_slot_advance

  6. Add parallel-aware hash joins.

  7. Fix typo.

  8. Allow --with-bonjour to work with non-macOS implementations of Bonjour.

  9. Mention need for --no-inc-recursive in rsync command