RE: Parallel Apply

Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>

From: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>

To: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, 'Amit Kapila' <amit.kapila16@gmail.com>

Cc: Tomas Vondra <tomas@vondra.me>, Dilip Kumar <dilipbalaut@gmail.com>, Andrei Lepikhov <lepihov@gmail.com>, wenhui qiu <qiuwenhuifx@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>

Date: 2026-04-16T16:59:25Z

Lists: pgsql-hackers

Attachments

v14-0009-Support-dependency-tracking-via-foreign-keys.patch (application/octet-stream) patch v14-0009
v14-0001-Introduce-internal-messages-to-track-dependencie.patch (application/octet-stream) patch v14-0001
v14-0002-Introduce-a-shared-hash-table-to-store-paralleli.patch (application/octet-stream) patch v14-0002
v14-0003-Introduce-a-local-hash-table-to-store-replica-id.patch (application/octet-stream) patch v14-0003
v14-0004-Parallel-apply-non-streaming-transactions.patch (application/octet-stream) patch v14-0004
v14-0005-support-2PC.patch (application/octet-stream) patch v14-0005
v14-0006-Track-dependencies-for-streamed-transactions.patch (application/octet-stream) patch v14-0006
v14-0007-Wait-applying-transaction-if-one-of-user-defined.patch (application/octet-stream) patch v14-0007
v14-0008-Support-dependency-tracking-via-local-unique-ind.patch (application/octet-stream) patch v14-0008

On Friday, April 17, 2026 12:05 AM Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com> wrote:
> 
> On Tuesday, April 14, 2026 9:00 PM Kuroda, Hayato/黒田 隼人
> <kuroda.hayato@fujitsu.com> wrote:
> >
> > Other comments were addressed accordingly, please see attached patch set.
> 
> I started reviewing patches 0001-0004 myself, aiming to add comments where
> the design is not straightforward and to identify and fix any clearly incorrect
> behavior.
> 
> Here is the updated patch set with the following improvements:
> 
> * Cosmetic changes in 0001-0004
> * Additional comments in 0001-0004
> * Code simplification by merging unnecessary static functions
> * Removal of function exports left over from the POC version that are no
>   longer needed
> * Got rid of XLogRecPtrIsInvalid()
> * Fixed buggy behavior in partial serialization mode, including:
>   1) The leader did not serialize the dependency on the last committed
>      transaction
>   2) The parallel apply worker could not identify internal messages in
>      spooled changes
>   3) An assertion failure in maybe_start_skipping_changes()
> * Added one test for serialization and restore non-streaming transactions in
>   0004.
> 
> Thanks to Kuroda-San for discussing these changes internally with me.

I noticed a CFbot failure caused by a missing identification of the internal
message code, sorry for that. Here are the updated patches to fix it.

Best Regards,
Hou zj