Re: BUG #15324: Non-deterministic behaviour from parallelised sub-query

Amit Kapila <amit.kapila16@gmail.com>

From: Amit Kapila <amit.kapila16@gmail.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Stephen Frost <sfrost@snowman.net>, Marko Tiikkaja <marko@joh.to>, Andres Freund <andres@anarazel.de>, Andrew Fletcher <andy@prestigedigital.com>, PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
Date: 2018-08-21T03:14:31Z
Lists: pgsql-bugs

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. Don't allow LIMIT/OFFSET clause within sub-selects to be pushed to workers.

  2. Back-patch "Fix parallel hash join path search."

  3. Prohibit pushing subqueries containing window function calculation to

  4. Fix parallel hash join path search.

On Mon, Aug 20, 2018 at 6:59 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Amit Kapila <amit.kapila16@gmail.com> writes:
>> On Mon, Aug 20, 2018 at 4:20 PM, Amit Kapila <amit.kapila16@gmail.com> wrote:
>>> On Thu, Aug 16, 2018 at 9:25 AM, Amit Kapila <amit.kapila16@gmail.com> wrote:
>> On Wed, Aug 15, 2018 at 4:40 PM, Stephen Frost <sfrost@snowman.net> wrote:
>>>> (b) Consider the presence of any window function calculation as
>>>> parallel-restricted operation.
>
>>> For this, we need to mark all the window functions like row_number,
>>> rank, dense_rank, etc as parallel-restricted.   Additionally, we also
>>> need to detect the presence of aggregate functions that act as window
>>> functions (when an OVER clause follows the call).   Attached patch
>>> treat_window_func_calc_parallel_restricted_v1 implements the fix.
>
>> As this patch changes the catalog contents, we need to bump catalog
>> version number.  However, I have left it for later once we get a
>> review and or testing of the patch.
>
> Sounds to me like you're using the wrong approach.  I would just consider
> any Agg or WindowFunc node as parallel-restricted regardless of the
> function it references.
>

I have below change in the patch which I think is on the lines what
you are describing, do you have something different in mind?

@@ -1197,6 +1197,19 @@ max_parallel_hazard_walker(Node *node,
max_parallel_hazard_context *context)
  }

  /*
+ * Treat window functions as parallel-restricted as the row ordering
+ * induced by them is non-deterministic.  We can relax this condition for
+ * cases where the row ordering can be deterministic like when there is
+ * an ORDER BY on the primary key, but those cases don't seem to be
+ * interesting enough to have additional checks.
+ */
+ if (IsA(node, WindowFunc))
+ {
+ if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
+ return true;
+ }

In addition to the above, I have marked all built-in window functions
as parallel-restricted.  I think even if we don't do that something
like above check should be sufficient, but OTOH, I don't see any
reason to keep the marking of such functions as parallel-safe.  Is
there a reason, why we shouldn't mark them as parallel-restricted?

-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com