Horton
Menu

Re: Add Pipelining support in psql

Daniel Verite <daniel@manitou-mail.org>

View thread
From: "Daniel Verite" <daniel@manitou-mail.org>
To: "Anthonin Bonnefoy" <anthonin.bonnefoy@datadoghq.com>
Cc: Michael Paquier <michael@paquier.xyz>, Jelte Fennema-Nio <postgres@jeltef.nl>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Date: 2025-03-04T12:32:38Z
Lists: pgsql-hackers 
	Anthonin Bonnefoy wrote:

> Another possible option would be to directly send the command without
> requiring an additional meta-command, like "SELECT 1 \bind". However,
> this would make it more painful to introduce new parameters, plus it
> makes the \bind and \bind_named inconsistent as it is normally
> required to send the result with a separate meta-command.

AFAIU the reason why \bind is required (even when there are no $N
parameters in the query) is to trigger the use of the extended query
protocol. This pre-dates the pipeline feature.

But when in a pipeline, we can't send queries with the simple query
protocol anyway, so a possible usability improvement would be to make
\bind optional in that case.

Concretely it's not possible currently to issue:
  \startpipeline
  select 1;
it causes the error:  "PQsendQuery not allowed in pipeline mode"

whereas this sequence does works:
  \startpipeline
  \bind
  select 1;
  \flushrequest
  \getresults

But if the code triggered the use of the extended query protocol
if \bind is in effect *or* a pipeline is active, then the first sequence
would  just push "select 1" into the pipeline.

This would have the advantage that, to submit into a pipeline
a pre-existing file with SQL commands separated with ";"  you don't have
to pre-process it to inject metacommands. Adding a \startpipeline at
the beginning and an \endpipeline at the end would be sufficient in the
cases that the user does not need the results before the end.

The \sendpipeline is not mandatory when ";" can be used to terminate
the queries. But it makes it clearer that the script wants
specifically to push into a pipeline, and it might accept specific
options in the future, whereas obviously ";" cannot.


Best regards,
-- 
Daniel Vérité 
https://postgresql.verite.pro/

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. psql: Fix assertion failures with pipeline mode

  2. psql: Rework TAP routine psql_fails_like() to define WAL sender context

  3. psql: Split extended query protocol meta-commands in --help=commands

  4. psql: Improve descriptions of \\flush[request] in --help

  5. psql: Fix incorrect status code returned by \getresults

  6. psql: Allow queries terminated by semicolons while in pipeline mode

  7. psql: Add \sendpipeline to send query buffers while in a pipeline

  8. psql: Fix memory leak with \gx used within a pipeline

  9. psql: Add pipeline status to prompt and some state variables

  10. Add more tests for utility commands in pipelines

  11. psql: Add support for pipelines

  12. Add braces for if block with large comment in psql's common.c