Re: PATCH: Batch/pipelining support for libpq

Daniel Verite <daniel@manitou-mail.org>

From: "Daniel Verite" <daniel@manitou-mail.org>
To: "Vaishnavi Prabakaran" <vaishnaviprabakaran@gmail.com>
Cc: "Michael Paquier" <michael.paquier@gmail.com>,"Prabakaran, Vaishnavi" <VaishnaviP@fast.au.fujitsu.com>,"Craig Ringer" <craig@2ndquadrant.com>,"Haribabu Kommi" <kommi.haribabu@gmail.com>,"Tsunakawa, Takayuki" <tsunakawa.takay@jp.fujitsu.com>,"Dmitry Igrishin" <dmitigr@gmail.com>,"Andres Freund" <andres@anarazel.de>,"PostgreSQL Hackers" <pgsql-hackers@postgresql.org>,"Manuel Kniep" <m.kniep@web.de>,"fujita.etsuro@lab.ntt.co.jp" <fujita.etsuro@lab.ntt.co.jp>,"Iwata, Aya" <iwata.aya@jp.fujitsu.com>
Date: 2017-03-07T16:52:35Z
Lists: pgsql-hackers

Attachments

	Vaishnavi Prabakaran wrote:

> Yes, I have created a new patch entry into the commitfest 2017-03 and
> attached the latest patch with this e-mail.

Please find attached a companion patch implementing the batch API in
pgbench, exposed as \beginbatch and \endbatch meta-commands
(without documentation).

The idea for now is to make it easier to exercise the API and test
how batching performs. I guess I'll submit the patch separately in
a future CF, depending on when/if the libpq patch goes in.

While developing this, I noted a few things with 0001-v4:

1. lack of initialization for count in PQbatchQueueCount.
Trivial fix:

--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -1874,7 +1874,7 @@ PQisBusy(PGconn *conn)
 int
 PQbatchQueueCount(PGconn *conn)
 {
-	int			count;
+	int			count = 0;
	PGcommandQueueEntry *entry;

2. misleading error message in PQexecStart. It gets called by a few other
functions than PQexec, such as PQprepare. As I understand it, the intent
here is to forbid the synchronous functions in batch mode, so this error
message should not single out PQexec.

@@ -1932,6 +2425,13 @@ PQexecStart(PGconn *conn)
	if (!conn)
		return false;
 
+	if (conn->asyncStatus == PGASYNC_QUEUED || conn->batch_status !=
PQBATCH_MODE_OFF)
+	{
+		printfPQExpBuffer(&conn->errorMessage,
+						  libpq_gettext("cannot
PQexec in batch mode\n"));
+		return false;
+	}
+

3. In relation to #2, PQsendQuery() is not forbidden in batch mode
although I don't think it can work with it, as it's based on the old
protocol. 


Best regards,
-- 
Daniel Vérité
PostgreSQL-powered mailer: http://www.manitou-mail.org
Twitter: @DanielVerite

Commits

  1. Add libpq pipeline mode support to pgbench

  2. Implement pipeline mode in libpq