Re: Asynchronous Append on postgres_fdw nodes.

Tom Lane <tgl@sss.pgh.pa.us>

From: Tom Lane <tgl@sss.pgh.pa.us>
To: Etsuro Fujita <etsuro.fujita@gmail.com>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>, Justin Pryzby <pryzby@telsasoft.com>, Andrey Lepikhov <a.lepikhov@postgrespro.ru>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2021-04-01T15:09:36Z
Lists: pgsql-hackers
Etsuro Fujita <etsuro.fujita@gmail.com> writes:
> Pushed.

The buildfarm points out that this fails under valgrind.
I easily reproduced it here:

==00:00:03:42.115 3410499== Syscall param epoll_wait(events) points to unaddressable byte(s)
==00:00:03:42.115 3410499==    at 0x58E926B: epoll_wait (epoll_wait.c:30)
==00:00:03:42.115 3410499==    by 0x7FC903: WaitEventSetWaitBlock (latch.c:1452)
==00:00:03:42.115 3410499==    by 0x7FC903: WaitEventSetWait (latch.c:1398)
==00:00:03:42.115 3410499==    by 0x6BF46C: ExecAppendAsyncEventWait (nodeAppend.c:1025)
==00:00:03:42.115 3410499==    by 0x6BF667: ExecAppendAsyncGetNext (nodeAppend.c:915)
==00:00:03:42.115 3410499==    by 0x6BF667: ExecAppend (nodeAppend.c:337)
==00:00:03:42.115 3410499==    by 0x6D49E4: ExecProcNode (executor.h:257)
==00:00:03:42.115 3410499==    by 0x6D49E4: ExecModifyTable (nodeModifyTable.c:2222)
==00:00:03:42.115 3410499==    by 0x6A87F2: ExecProcNode (executor.h:257)
==00:00:03:42.115 3410499==    by 0x6A87F2: ExecutePlan (execMain.c:1531)
==00:00:03:42.115 3410499==    by 0x6A87F2: standard_ExecutorRun (execMain.c:350)
==00:00:03:42.115 3410499==    by 0x82597F: ProcessQuery (pquery.c:160)
==00:00:03:42.115 3410499==    by 0x825BE9: PortalRunMulti (pquery.c:1267)
==00:00:03:42.115 3410499==    by 0x826826: PortalRun (pquery.c:779)
==00:00:03:42.115 3410499==    by 0x82291E: exec_simple_query (postgres.c:1185)
==00:00:03:42.115 3410499==    by 0x823F3E: PostgresMain (postgres.c:4415)
==00:00:03:42.115 3410499==    by 0x79BAC1: BackendRun (postmaster.c:4483)
==00:00:03:42.115 3410499==    by 0x79BAC1: BackendStartup (postmaster.c:4205)
==00:00:03:42.115 3410499==    by 0x79BAC1: ServerLoop (postmaster.c:1737)
==00:00:03:42.115 3410499==  Address 0x10d10628 is 7,960 bytes inside a recently re-allocated block of size 8,192 alloc'd
==00:00:03:42.115 3410499==    at 0x4C30F0B: malloc (vg_replace_malloc.c:307)
==00:00:03:42.115 3410499==    by 0x94F9EA: AllocSetAlloc (aset.c:919)
==00:00:03:42.115 3410499==    by 0x957BAF: MemoryContextAlloc (mcxt.c:809)
==00:00:03:42.115 3410499==    by 0x958CC0: MemoryContextStrdup (mcxt.c:1179)
==00:00:03:42.115 3410499==    by 0x516AE4: untransformRelOptions (reloptions.c:1336)
==00:00:03:42.115 3410499==    by 0x6E6ADF: GetForeignTable (foreign.c:273)
==00:00:03:42.115 3410499==    by 0xF3BD470: postgresBeginForeignScan (postgres_fdw.c:1479)
==00:00:03:42.115 3410499==    by 0x6C2E83: ExecInitForeignScan (nodeForeignscan.c:236)
==00:00:03:42.115 3410499==    by 0x6AF893: ExecInitNode (execProcnode.c:283)
==00:00:03:42.115 3410499==    by 0x6C0007: ExecInitAppend (nodeAppend.c:232)
==00:00:03:42.115 3410499==    by 0x6AFA37: ExecInitNode (execProcnode.c:180)
==00:00:03:42.115 3410499==    by 0x6D533A: ExecInitModifyTable (nodeModifyTable.c:2575)

==00:00:03:44.907 3410499== Syscall param epoll_wait(events) points to unaddressable byte(s)
==00:00:03:44.907 3410499==    at 0x58E926B: epoll_wait (epoll_wait.c:30)
==00:00:03:44.907 3410499==    by 0x7FC903: WaitEventSetWaitBlock (latch.c:1452)
==00:00:03:44.907 3410499==    by 0x7FC903: WaitEventSetWait (latch.c:1398)
==00:00:03:44.907 3410499==    by 0x6BF46C: ExecAppendAsyncEventWait (nodeAppend.c:1025)
==00:00:03:44.907 3410499==    by 0x6BF718: ExecAppend (nodeAppend.c:370)
==00:00:03:44.907 3410499==    by 0x6D49E4: ExecProcNode (executor.h:257)
==00:00:03:44.907 3410499==    by 0x6D49E4: ExecModifyTable (nodeModifyTable.c:2222)
==00:00:03:44.907 3410499==    by 0x6A87F2: ExecProcNode (executor.h:257)
==00:00:03:44.907 3410499==    by 0x6A87F2: ExecutePlan (execMain.c:1531)
==00:00:03:44.907 3410499==    by 0x6A87F2: standard_ExecutorRun (execMain.c:350)
==00:00:03:44.907 3410499==    by 0x82597F: ProcessQuery (pquery.c:160)
==00:00:03:44.907 3410499==    by 0x825BE9: PortalRunMulti (pquery.c:1267)
==00:00:03:44.907 3410499==    by 0x826826: PortalRun (pquery.c:779)
==00:00:03:44.907 3410499==    by 0x82291E: exec_simple_query (postgres.c:1185)
==00:00:03:44.907 3410499==    by 0x823F3E: PostgresMain (postgres.c:4415)
==00:00:03:44.907 3410499==    by 0x79BAC1: BackendRun (postmaster.c:4483)
==00:00:03:44.907 3410499==    by 0x79BAC1: BackendStartup (postmaster.c:4205)
==00:00:03:44.907 3410499==    by 0x79BAC1: ServerLoop (postmaster.c:1737)
==00:00:03:44.907 3410499==  Address 0x1093fdd8 is 2,904 bytes inside a recently re-allocated block of size 16,384 alloc'd
==00:00:03:44.907 3410499==    at 0x4C30F0B: malloc (vg_replace_malloc.c:307)
==00:00:03:44.907 3410499==    by 0x94F9EA: AllocSetAlloc (aset.c:919)
==00:00:03:44.907 3410499==    by 0x958233: palloc (mcxt.c:964)
==00:00:03:44.907 3410499==    by 0x69C400: ExprEvalPushStep (execExpr.c:2310)
==00:00:03:44.907 3410499==    by 0x69C541: ExecPushExprSlots (execExpr.c:2490)
==00:00:03:44.907 3410499==    by 0x69C580: ExecInitExprSlots (execExpr.c:2445)
==00:00:03:44.907 3410499==    by 0x69F0DD: ExecInitQual (execExpr.c:231)
==00:00:03:44.907 3410499==    by 0x6D80EF: ExecInitSeqScan (nodeSeqscan.c:172)
==00:00:03:44.907 3410499==    by 0x6AF9CE: ExecInitNode (execProcnode.c:208)
==00:00:03:44.907 3410499==    by 0x6C0007: ExecInitAppend (nodeAppend.c:232)
==00:00:03:44.907 3410499==    by 0x6AFA37: ExecInitNode (execProcnode.c:180)
==00:00:03:44.907 3410499==    by 0x6D533A: ExecInitModifyTable (nodeModifyTable.c:2575)
==00:00:03:44.907 3410499== 

Sorta looks like something is relying on a pointer into the relcache
to be valid for longer than it can safely rely on that.  The
CLOBBER_CACHE_ALWAYS animals will probably be unhappy too, but
they are slower than valgrind.

(Note that the test case appears to succeed, you have to notice that
the backend crashed after exiting.)

			regards, tom lane



Commits

  1. Improve comments for trivial_subqueryscan().

  2. Disable asynchronous execution if using gating Result nodes.

  3. Allow asynchronous execution in more cases.

  4. Doc: Further update documentation for asynchronous execution.

  5. Fix rescanning of async-aware Append nodes.

  6. Doc: Update documentation for asynchronous execution.

  7. Fix EXPLAIN ANALYZE for async-capable nodes.

  8. Minor code cleanup in asynchronous execution support.

  9. Adjust input value to WaitEventSetWait() in ExecAppendAsyncEventWait().

  10. Add support for asynchronous execution.