Re: PATCH: Batch/pipelining support for libpq

Daniel Verite <daniel@manitou-mail.org>

From: "Daniel Verite" <daniel@manitou-mail.org>
To: "Vaishnavi Prabakaran" <vaishnaviprabakaran@gmail.com>
Cc: "Michael Paquier" <michael.paquier@gmail.com>,"Prabakaran, Vaishnavi" <VaishnaviP@fast.au.fujitsu.com>,"Craig Ringer" <craig@2ndquadrant.com>,"Haribabu Kommi" <kommi.haribabu@gmail.com>,"Tsunakawa, Takayuki" <tsunakawa.takay@jp.fujitsu.com>,"Dmitry Igrishin" <dmitigr@gmail.com>,"Andres Freund" <andres@anarazel.de>,"PostgreSQL Hackers" <pgsql-hackers@postgresql.org>,"Manuel Kniep" <m.kniep@web.de>,"fujita.etsuro@lab.ntt.co.jp" <fujita.etsuro@lab.ntt.co.jp>,"Iwata, Aya" <iwata.aya@jp.fujitsu.com>
Date: 2017-03-10T13:52:05Z
Lists: pgsql-hackers

Attachments

  Hi,

I notice that PQsetSingleRowMode() doesn't work when getting batch results.  

The function is documented as:
" int PQsetSingleRowMode(PGconn *conn);

  This function can only be called immediately after PQsendQuery or one
  of its sibling functions, before any other operation on the connection
  such as PQconsumeInput or PQgetResult"

But PQbatchQueueProcess() unconditionally clears conn->singleRowMode,
so whatever it was when sending the query gets lost, and besides
other queries might have been submitted in the meantime.

Also if trying to set that mode when fetching like this

     while (QbatchQueueProcess(conn)) {
       r = PQsetSingleRowMode(conn);
       if (r!=1) {
	  fprintf(stderr, "PQsetSingleRowMode() failed");
       }
       .. 

it might work the first time, but on the next iterations, conn->asyncStatus
might be PGASYNC_READY, which is a failure condition for
PQsetSingleRowMode(), so that won't do.

ISTM that the simplest fix would be that when in batch mode,
PQsetSingleRowMode() should register that the last submitted query
does request that mode, and when later QbatchQueueProcess dequeues
the batch entry for the corresponding query, this flag would be popped off
and set as the current mode.

Please find attached the suggested fix, against the v5 of the patch.

Best regards,
-- 
Daniel Vérité
PostgreSQL-powered mailer: http://www.manitou-mail.org
Twitter: @DanielVerite

Commits

  1. Add libpq pipeline mode support to pgbench

  2. Implement pipeline mode in libpq