Re: [HACKERS] PATCH: Batch/pipelining support for libpq

Daniel Verite <daniel@manitou-mail.org>

From: "Daniel Verite" <daniel@manitou-mail.org>
To: "Kyotaro HORIGUCHI" <horiguchi.kyotaro@lab.ntt.co.jp>
Cc: vaishnaviprabakaran@gmail.com,michael.paquier@gmail.com,daniel@yesql.se,craig@2ndquadrant.com,andres@anarazel.de,david@pgmasters.net,VaishnaviP@fast.au.fujitsu.com,kommi.haribabu@gmail.com,tsunakawa.takay@jp.fujitsu.com,dmitigr@gmail.com,pgsql-hackers@postgresql.org,m.kniep@web.de,fujita.etsuro@lab.ntt.co.jp,iwata.aya@jp.fujitsu.com
Date: 2018-03-23T13:18:09Z
Lists: pgsql-hackers
	Kyotaro HORIGUCHI wrote:

> A disucssion on psql batch mode was held in another branch of
> this thread. How do we treat that?

There's a batch mode for pgbench in a patch posted in [1],
with \beginbatch and \endbatch commands, but nothing
for psql AFAICS.
psql is more complicated because currently it uses a 
blocking PQexec() call at its core. Craig mentioned psql
integration in [2] and [3].
Also a script can have inter-query dependencies such as in
   insert into table(...) returning id \gset
   update othertable set col= :id where ...;
which is a problem in batch mode, as we don't want
to send the update before the right value for :id is
known. Whether we want to support these dependencies
and how needs discussion.
For instance we might not support them at all, or
create a synchronization command that collects
all results of queries sent so far, or do it implicitly
when a variable is injected into a query...
This looks like substantial work that might be best
done separately from the libpq patch.

[1]
https://www.postgresql.org/message-id/b4e34135-2bd9-4b8a-94ca-27d760da26d7@manitou-mail.org
[2]
https://www.postgresql.org/message-id/CAMsr+YGLhaDkjymLuNVQy4MrSKQoA=F1vO=aN8XQf30N=aQuVA@mail.gmail.com
[3]
https://www.postgresql.org/message-id/CAMsr+YE6BK4iAaQz=nY3xDnbLhnNZ_4tp-PTJqbNNpsZMgoo8Q@mail.gmail.com


Best regards,
-- 
Daniel Vérité
PostgreSQL-powered mailer: http://www.manitou-mail.org
Twitter: @DanielVerite


Commits

  1. Add libpq pipeline mode support to pgbench

  2. Implement pipeline mode in libpq