Re: Enabling parallelism for queries coming from SQL or other PL functions

Amit Kapila <amit.kapila16@gmail.com>

From: Amit Kapila <amit.kapila16@gmail.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Dilip Kumar <dilipbalaut@gmail.com>, Rafia Sabih <rafia.sabih@enterprisedb.com>, PostgreSQL Developers <pgsql-hackers@postgresql.org>
Date: 2017-03-02T10:24:38Z
Lists: pgsql-hackers
On Thu, Mar 2, 2017 at 3:50 PM, Robert Haas <robertmhaas@gmail.com> wrote:
> On Tue, Feb 28, 2017 at 5:25 PM, Amit Kapila <amit.kapila16@gmail.com> wrote:
>> When such a function (that contains statements which have parallel
>> plans) is being executed as part of another parallel plan, it can
>> allow spawning workers unboundedly.   Assume a query like  select *
>> from t1 where c1 < func1(),  this can use parallel scan for t1 and
>> then in master backend, during partial scan of t1, it can again spawn
>> new set of workers for queries inside func1(), this can happen
>> multiple times if parallel query inside func1() again calls some other
>> function func2() which has parallel query.  Now, this might be okay,
>> but today such a situation doesn't exist that Gather execution can
>> invoke another Gather node, so it is worth to consider if we want to
>> allow it.
>
> If we want to prohibit that, the check in standard_planner can be
> changed from !IsParallelWorker() to !IsInParallelMode(), but I'm not
> 100% sure whether that's an improvement or not.
>

I am not sure how you can achieve that by just changing
standard_planner() code, because the plans of statements inside pl can
be cached in which case it will not try to regenerate the plan.

>  I would be inclined
> to leave it alone unless we get several votes to change it.
>

Okay, not a problem.


-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com


Commits

  1. Improve access to parallel query from procedural languages.

  2. plpgsql: Don't generate parallel plans for RETURN QUERY.

  3. Allow for parallel execution whenever ExecutorRun() is done only once.