Re: Asynchronous Append on postgres_fdw nodes.

Andrei Lepikhov <a.lepikhov@postgrespro.ru>

From: Andrey Lepikhov <a.lepikhov@postgrespro.ru>
To: Etsuro Fujita <etsuro.fujita@gmail.com>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>, Justin Pryzby <pryzby@telsasoft.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2021-05-11T02:58:10Z
Lists: pgsql-hackers
On 10/5/21 08:03, Etsuro Fujita wrote:
> On Fri, May 7, 2021 at 7:32 PM Andrey Lepikhov
> I think a simple fix for this would be just remove the check whether
> the instr->running flag is set or not in InstrUpdateTupleCount().
> Attached is an updated patch, in which I also updated a comment in
> execnodes.h and docs in fdwhandler.sgml to match the code in
> nodeAppend.c, and fixed typos in comments in nodeAppend.c.
Your patch fixes the problem. But I found two more problems:

EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF) 
 
 
SELECT * FROM ( 
 
 
               (SELECT * FROM f1) 
 
 
                              UNION ALL 
 
 
                                     (SELECT * FROM f2) 
 
 
                                                    UNION ALL 
 
 
                                                           (SELECT * 
FROM l3) 
 
                                                                    ) q1 
LIMIT 6709;
                           QUERY PLAN
--------------------------------------------------------------
  Limit (actual rows=6709 loops=1)
    ->  Append (actual rows=6709 loops=1)
          ->  Async Foreign Scan on f1 (actual rows=1 loops=1)
          ->  Async Foreign Scan on f2 (actual rows=1 loops=1)
          ->  Seq Scan on l3 (actual rows=6708 loops=1)

Here we scan 6710 tuples at low level but appended only 6709. Where did 
we lose one tuple?

2.
SELECT * FROM (
	(SELECT * FROM f1) 
 
 
                UNION ALL
	(SELECT * FROM f2) 
 
 
                UNION ALL
	(SELECT * FROM f3 WHERE a > 0)
) q1 LIMIT 3000;
                           QUERY PLAN
--------------------------------------------------------------
  Limit (actual rows=3000 loops=1)
    ->  Append (actual rows=3000 loops=1)
          ->  Async Foreign Scan on f1 (actual rows=0 loops=1)
          ->  Async Foreign Scan on f2 (actual rows=0 loops=1)
          ->  Foreign Scan on f3 (actual rows=3000 loops=1)

Here we give preference to the synchronous scan. Why?

-- 
regards,
Andrey Lepikhov
Postgres Professional



Commits

  1. Improve comments for trivial_subqueryscan().

  2. Disable asynchronous execution if using gating Result nodes.

  3. Allow asynchronous execution in more cases.

  4. Doc: Further update documentation for asynchronous execution.

  5. Fix rescanning of async-aware Append nodes.

  6. Doc: Update documentation for asynchronous execution.

  7. Fix EXPLAIN ANALYZE for async-capable nodes.

  8. Minor code cleanup in asynchronous execution support.

  9. Adjust input value to WaitEventSetWait() in ExecAppendAsyncEventWait().

  10. Add support for asynchronous execution.