Re: [HACKERS] PATCH: Batch/pipelining support for libpq
Alvaro Herrera <alvherre@2ndquadrant.com>
From: Alvaro Herrera <alvherre@2ndquadrant.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Justin Pryzby <pryzby@telsasoft.com>, pgsql-hackers@lists.postgresql.org, Craig Ringer <craig.ringer@enterprisedb.com>, Matthieu Garrigues <matthieu.garrigues@gmail.com>, Zhihong Yu <zyu@yugabyte.com>, Álvaro Herrera <alvherre@alvh.no-ip.org>, Andres Freund <andres@anarazel.de>, Aya Iwata <iwata.aya@jp.fujitsu.com>, Daniel Vérité <daniel@manitou-mail.org>, "David G. Johnston" <david.g.johnston@gmail.com>, Kirk Jamison <k.jamison@fujitsu.com>, Michael Paquier <michael.paquier@gmail.com>, Nikhil Sontakke <nikhils@2ndquadrant.com>, Vaishnavi Prabakaran <VaishnaviP@fast.au.fujitsu.com>
Date: 2021-03-15T18:54:54Z
Lists: pgsql-hackers
Attachments
- v37-0001-Implement-pipeline-mode-in-libpq.patch (text/x-diff)
Here's what seems a final version of the patch. I renamed one more
function: PQsendPipeline is now PQpipelineSync. I also reworded the
docs in a couple of places, added a few tests to the pgbench patch, and
made it work.
Note the pgbench results in pipeline mode:
./pgbench -r -Mextended -n -f /home/alvherre/Code/pgsql-build/pipeline/src/bin/pgbench/tmp_check/t_001_pgbench_with_server_main_data/001_pgbench_pipeline -c 100 -t10000
pgbench (PostgreSQL) 14.0
transaction type: /home/alvherre/Code/pgsql-build/pipeline/src/bin/pgbench/tmp_check/t_001_pgbench_with_server_main_data/001_pgbench_pipeline
scaling factor: 1
query mode: extended
number of clients: 100
number of threads: 1
number of transactions per client: 10000
number of transactions actually processed: 1000000/1000000
latency average = 2.316 ms
initial connection time = 113.859 ms
tps = 43182.438635 (without initial connection time)
statement latencies in milliseconds:
0.000 \startpipeline
0.000 select 1;
0.000 select 1;
0.000 select 1;
0.000 select 1;
0.000 select 1;
0.000 select 1;
0.000 select 1;
0.000 select 1;
0.000 select 1;
0.000 select 1;
1.624 \endpipeline
If I just replace the \startpipeline and \endpipeline lines with BEGIN
and COMMIT respectively, I get this:
tps = 10220.259051 (without initial connection time)
0.830 begin;
0.765 select 1;
0.752 select 1;
0.753 select 1;
0.755 select 1;
0.754 select 1;
0.755 select 1;
0.757 select 1;
0.756 select 1;
0.756 select 1;
0.756 select 1;
0.750 commit;
Yes, you could say that this is a liiiitle bit unfair -- but it seems
quite impressive nonetheless.
--
Álvaro Herrera 39°49'30"S 73°17'W
Commits
-
Add libpq pipeline mode support to pgbench
- 9aa491abbf07 14.0 landed
-
Implement pipeline mode in libpq
- acb7e4eb6b1c 14.0 landed