Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

David Zhang <david.zhang@highgo.ca>

From: David Zhang <david.zhang@highgo.ca>
To: Etsuro Fujita <etsuro.fujita@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Cc: Fujii Masao <masao.fujii@oss.nttdata.com>
Date: 2021-11-01T22:46:39Z
Lists: pgsql-hackers
> I evaluated the effectiveness of the patch using a simple
> multi-statement transaction:
>
> BEGIN;
> SAVEPOINT s;
> INSERT INTO ft1 VALUES (10, 10);
> INSERT INTO ft2 VALUES (20, 20);
> RELEASE SAVEPOINT s;
> COMMIT;
>
> where ft1 and ft2 are foreign tables created on different foreign
> servers hosted on different machines.  I ran the transaction five
> times using the patch with the option enabled/disabled, and measured
> the latencies for the RELEASE and COMMIT commands in each run.  The
> average latencies for these commands over the five runs are:
>
> * RELEASE
>    parallel_commit=0: 0.385 ms
>    parallel_commit=1: 0.221 ms
>
> * COMMIT
>    parallel_commit=0: 1.660 ms
>    parallel_commit=1: 0.861 ms
>
> With the option enabled, the average latencies for both commands are
> reduced significantly!
Followed your instructions, I performed some basic tests to compare the 
performance between before and after. In my testing environment (two 
foreign servers on the same local machine), the performance varies, 
sometimes the time spent on RELEASE and COMMIT without patch are close 
to after patched, but seems it always perform better after patched. Then 
I ran a 1-millions tuples insert, 5 times average is something like below,

Before
     RELEASE 0.171 ms, COMMIT 1.861 ms

After
     RELEASE 0.147 ms, COMMIT 1.305 ms

Best regards,
-- 
David

Software Engineer
Highgo Software Inc. (Canada)
www.highgo.ca



Commits

  1. postgres_fdw: Add support for parallel abort.

  2. Refine the definition of page-level freezing.

  3. postgres_fdw: Update comments in make_new_connection().

  4. postgres_fdw: Minor cleanup for pgfdw_abort_cleanup().

  5. postgres_fdw: Add support for parallel commit.

  6. Fix ruleutils.c's dumping of whole-row Vars in more contexts.

  7. postgres_fdw: Refactor transaction rollback code to avoid code duplication.