Re: non-bulk inserts and tuple routing

(2018/02/09 14:32), Amit Langote wrote:
> I had mistakenly tagged these patches v24, but they were actually supposed
> to be v5.  So the attached updated patch is tagged v6.

OK.

> On 2018/02/07 19:36, Etsuro Fujita wrote:
>>> (2018/02/05 14:34), Amit Langote wrote:
>>>> The code in tupconv_map_for_subplan() currently assumes that it can rely
>>>> on all leaf partitions having been initialized.
>>
>> On reflection I noticed this analysis is not 100% correct; I think what
>> that function actually assumes is that all sublplans' partitions have
>> already been initialized, not all leaf partitions.
>
> Yes, you're right.
>
>>>> Since we're breaking that
>>>> assumption with this proposal, that needed to be changed. So the patch
>>>> contained some refactoring to make it not rely on that assumption.
>>
>> I don't think we really need this refactoring because since that as in
>> another patch you posted, we could initialize all subplans' partitions in
>> ExecSetupPartitionTupleRouting, I think tupconv_map_for_subplan could be
>> called without any changes to that function because of what I said above.
>
> What my previous approach failed to consider is that in the update case,
> we'd already have ResultRelInfo's for some of the leaf partitions
> initialized, which could be saved into proute->partitions array right away.
>
> Updated patch does things that way, so all the changes I had proposed to
> tupconv_map_for_subplan are rendered unnecessary.

OK, thanks for the updated patch!

>> Here are comments for the other patch (patch
>> v24-0002-During-tuple-routing-initialize-per-partition-ob.patch):
>>
>> o On changes to ExecSetupPartitionTupleRouting:
>>
>> * The comment below wouldn't be correct; no ExecInitResultRelInfo in the
>> patch.
>>
>> -   proute->partitions = (ResultRelInfo **) palloc(proute->num_partitions *
>> -                                                  sizeof(ResultRelInfo *));
>> +   /*
>> +    * Actual ResultRelInfo's and TupleConversionMap's are allocated in
>> +    * ExecInitResultRelInfo().
>> +    */
>> +   proute->partitions = (ResultRelInfo **) palloc0(proute->num_partitions *
>> +                                                   sizeof(ResultRelInfo
> *));
>
> I removed the comment altogether, as the comments elsewhere make the point
> clear.
>
>> * The patch removes this from the initialization step for a subplan's
>> partition, but I think it would be better to keep this here because I
>> think it's a good thing to put the initialization stuff together into one
>> place.
>>
>> -               /*
>> -                * This is required in order to we convert the partition's
>> -                * tuple to be compatible with the root partitioned table's
>> -                * tuple descriptor.  When generating the per-subplan result
>> -                * rels, this was not set.
>> -                */
>> -               leaf_part_rri->ri_PartitionRoot = rel;
>
> It wasn't needed here with the previous approach, because we didn't touch
> any ResultRelInfo's in ExecSetupPartitionTupleRouting, but I've added it
> back in the updated patch.
>
>> * I think it would be better to keep this comment here.
>>
>> -               /* Remember the subplan offset for this ResultRelInfo */
>
> Fixed.
>
>> * Why is this removed from that initialization?
>>
>> -       proute->partitions[i] = leaf_part_rri;
>
> Because of the old approach.  Now it's back in.
>
>> o On changes to ExecInitPartitionInfo:
>>
>> * I don't understand the step starting from this, but I'm wondering if
>> that step can be removed by keeping the above setup of proute->partitions
>> for the subplan's partition in ExecSetupPartitionTupleRouting.
>>
>> +   /*
>> +    * If we are doing tuple routing for update, try to reuse the
>> +    * per-subplan resultrel for this partition that ExecInitModifyTable()
>> +    * might already have created.
>> +    */
>> +   if (mtstate&&  mtstate->operation == CMD_UPDATE)
>
> Done, as mentioned above.
>
> On 2018/02/08 19:16, Etsuro Fujita wrote:
>> Here are some minor comments:
>>
>> o On changes to ExecInsert
>>
>> * This might be just my taste, but I think it would be better to (1)
>> change ExecInitPartitionInfo so that it creates and returns a
>> newly-initialized ResultRelInfo for an initialized partition, and (2)
>> rewrite this bit:
>>
>> +       /* Initialize partition info, if not done already. */
>> +       ExecInitPartitionInfo(mtstate, resultRelInfo, proute, estate,
>> +                             leaf_part_index);
>> +
>>          /*
>>           * Save the old ResultRelInfo and switch to the one corresponding to
>>           * the selected partition.
>>           */
>>          saved_resultRelInfo = resultRelInfo;
>>          resultRelInfo = proute->partitions[leaf_part_index];
>> +       Assert(resultRelInfo != NULL);
>>
>> to something like this (I would say the same thing to the copy.c changes):
>>
>>      /*
>>       * Save the old ResultRelInfo and switch to the one corresponding to
>>       * the selected partition.
>>       */
>>      saved_resultRelInfo = resultRelInfo;
>>      resultRelInfo = proute->partitions[leaf_part_index];
>>      if (resultRelInfo == NULL);
>>      {
>>          /* Initialize partition info. */
>>          resultRelInfo = ExecInitPartitionInfo(mtstate,
>>                                                saved_resultRelInfo,
>>                                                proute,
>>                                                estate,
>>                                                leaf_part_index);
>>      }
>>
>> This would make ExecInitPartitionInfo more simple because it can assume
>> that the given partition has not been initialized yet.
>
> Agree that it's much better to do it this way.  Done.

Thanks for all those changes!

>> o On changes to execPartition.h
>>
>> * Please add a brief decsription about partition_oids to the comments for
>> this struct.
>>
>> @@ -91,6 +91,7 @@ typedef struct PartitionTupleRouting
>>   {
>>      PartitionDispatch *partition_dispatch_info;
>>      int         num_dispatch;
>> +   Oid        *partition_oids;
>
> Done.

Thanks, but one thing I'm wondering is: do we really need this array?  I 
think we could store into PartitionTupleRouting the list of oids 
returned by RelationGetPartitionDispatchInfo in 
ExecSetupPartitionTupleRouting, instead.  Sorry, I should have commented 
this in a previous email, but what do you think about that?

Here are other comments:

o On changes to ExecSetupPartitionTupleRouting:

* This is nitpicking, but it would be better to define partrel and 
part_tupdesc within the if (update_rri_index < num_update_rri && 
RelationGetRelid(update_rri[update_rri_index].ri_RelationDesc) == 
leaf_oid) block.

-               ResultRelInfo *leaf_part_rri;
+               ResultRelInfo *leaf_part_rri = NULL;
                 Relation        partrel = NULL;
                 TupleDesc       part_tupdesc;
                 Oid                     leaf_oid = lfirst_oid(cell);

* Do we need this?  For a leaf partition that is already present in the 
subplan resultrels, the partition's indices (if any) would have already 
been opened.

+                               /*
+                                * Open partition indices.  We wouldn't 
need speculative
+                                * insertions though.
+                                */
+                               if 
(leaf_part_rri->ri_RelationDesc->rd_rel->relhasindex &&
+ 
leaf_part_rri->ri_IndexRelationDescs == NULL)
+                                       ExecOpenIndices(leaf_part_rri, 
false);

I'll look at the patch a bit more early next week, but other than that, 
the patch looks fairly in good shape to me.

Best regards,
Etsuro Fujita