Horton

Re: Asynchronous Append on postgres_fdw nodes.

Kyotaro Horiguchi <horikyota.ntt@gmail.com>

View thread View on postgr.es
From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
To: etsuro.fujita@gmail.com
Cc: pryzby@telsasoft.com, a.lepikhov@postgrespro.ru, movead.li@highgo.ca, pgsql-hackers@lists.postgresql.org
Date: 2021-02-12T08:30:43Z
Lists: pgsql-hackers 
At Wed, 10 Feb 2021 21:31:15 +0900, Etsuro Fujita <etsuro.fujita@gmail.com> wrote in 
> On Wed, Feb 10, 2021 at 7:31 PM Etsuro Fujita <etsuro.fujita@gmail.com> wrote:
> > Attached is an updated version of the patch.  Sorry for the delay.
> 
> I noticed that I forgot to add new files.  :-(.  Please find attached
> an updated patch.

Thanks for the new version.

It seems too specific to async Append so I look it as a PoC of the
mechanism.

It creates a hash table that keyed by connection umid to record
planids run on the connection, triggerd by core planner via a dedicate
API function.  It seems to me that ConnCacheEntry.state can hold that
and the hash is not needed at all.

| postgresReconsiderAsyncForeignScan(ForeignScanState *node, AsyncContext *acxt)
| {
| ...
|     /*
| 	 * If the connection used for the ForeignScan node is used in other parts
| 	 * of the query plan tree except async subplans of the parent Append node,
| 	 * disable async execution of the ForeignScan node.
| 	 */
| 	if (!bms_is_subset(fsplanids, asyncplanids))
| 		return false;

This would be a reasonable restriction.

| 	/*
| 	 * If the subplans of the Append node are all async-capable, and use the
| 	 * same connection, then we won't execute them asynchronously.
| 	 */
| 	if (requestor->as_nasyncplans == requestor->as_nplans &&
| 		!bms_nonempty_difference(asyncplanids, fsplanids))
| 		return false;

It is the correct restiction?  I understand that the currently
intending restriction is one connection accepts at most one FDW-scan
node. This looks somethig different...

(Sorry, time's up for now.)

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

Commits

  1. Improve comments for trivial_subqueryscan().

  2. Disable asynchronous execution if using gating Result nodes.

  3. Allow asynchronous execution in more cases.

  4. Doc: Further update documentation for asynchronous execution.

  5. Fix rescanning of async-aware Append nodes.

  6. Doc: Update documentation for asynchronous execution.

  7. Fix EXPLAIN ANALYZE for async-capable nodes.

  8. Minor code cleanup in asynchronous execution support.

  9. Adjust input value to WaitEventSetWait() in ExecAppendAsyncEventWait().

  10. Add support for asynchronous execution.