Horton

Re: Asynchronous Append on postgres_fdw nodes.

Etsuro Fujita <etsuro.fujita@gmail.com>

View thread View on postgr.es
From: Etsuro Fujita <etsuro.fujita@gmail.com>
To: "Andrey V. Lepikhov" <a.lepikhov@postgrespro.ru>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>, Justin Pryzby <pryzby@telsasoft.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2021-04-27T12:27:05Z
Lists: pgsql-hackers 
On Mon, Apr 26, 2021 at 3:01 PM Andrey V. Lepikhov
<a.lepikhov@postgrespro.ru> wrote:
> While studying the capabilities of AsyncAppend, I noticed an
> inconsistency with the cost model of the optimizer:

> Here I see two problems:
> 1. Cost of an AsyncAppend is the same as cost of an Append. But
> execution time of the AsyncAppend for three remote partitions has more
> than halved.
> 2. Cost of an AsyncAppend looks as a sum of the child ForeignScan costs.

Yeah, we don’t adjust the cost for async Append; it’s the same as that
for sync Append.  But I don’t see any issue as-is, either.  (It’s not
that easy to adjust the cost to an appropriate value in the case of
postgres_fdw, because in that case the cost would vary depending on
which connections are used for scanning foreign tables [1].)

> I haven't ideas why it may be a problem right now. But I can imagine
> that it may be a problem in future if we have alternative paths: complex
> pushdown in synchronous mode (a few rows to return) or simple
> asynchronous append with a large set of rows to return.

Yeah, I think it’s better if we could consider async append paths and
estimate the costs for them accurately at path-creation time, not
plan-creation time, because that would make it possible to use async
execution in more cases, as you pointed out.  But I left that for
future work, because I wanted to make the first cut simple.

Thanks for the review!

Best regards,
Etsuro Fujita

[1] https://www.postgresql.org/message-id/CAPmGK15i-OyCesd369P8zyBErjN_T18zVYu27714bf_L%3DCOXew%40mail.gmail.com

Commits

  1. Improve comments for trivial_subqueryscan().

  2. Disable asynchronous execution if using gating Result nodes.

  3. Allow asynchronous execution in more cases.

  4. Doc: Further update documentation for asynchronous execution.

  5. Fix rescanning of async-aware Append nodes.

  6. Doc: Update documentation for asynchronous execution.

  7. Fix EXPLAIN ANALYZE for async-capable nodes.

  8. Minor code cleanup in asynchronous execution support.

  9. Adjust input value to WaitEventSetWait() in ExecAppendAsyncEventWait().

  10. Add support for asynchronous execution.