Horton
Menu

Re: Add Pipelining support in psql

Daniel Verite <daniel@manitou-mail.org>

View thread
From: "Daniel Verite" <daniel@manitou-mail.org>
To: "Anthonin Bonnefoy" <anthonin.bonnefoy@datadoghq.com>
Cc: Michael Paquier <michael@paquier.xyz>, Jelte Fennema-Nio <postgres@jeltef.nl>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Date: 2025-03-05T14:25:12Z
Lists: pgsql-hackers 
	Anthonin Bonnefoy wrote:

> So if I understand correctly, you want to automatically convert a
> simple query into an extended query when we're within a pipeline. That
> would be doable with:
> 
> --- a/src/bin/psql/common.c
> +++ b/src/bin/psql/common.c
> @@ -1668,7 +1668,16 @@ ExecQueryAndProcessResults(const char *query,
>                        }
>                        break;
>                case PSQL_SEND_QUERY:
> -                       success = PQsendQuery(pset.db, query);
> +                       if (PQpipelineStatus(pset.db) != PQ_PIPELINE_OFF) {
> +                               success = PQsendQueryParams(pset.db, query,
> +
> pset.bind_nparams, NULL,
> +                                                               (const
> char *const *) pset.bind_params,
> +                                                               NULL, NULL,
> 0);
> +                               if (success)
> +                                       pset.piped_commands++;
> +                       }
> +                       else
> +                               success = PQsendQuery(pset.db, query);
>                        break;
>        }

Yes, except that the bind parameters need to be cleared after this,
as done in clean_extended_state()

> I do see the idea to make it easier to convert existing scripts into
> using pipelining. The main focus of the initial implementation was
> more on protocol regression tests with psql, so that's not necessarily
> something I had in mind.

Understood. Yet pipelining can accelerate considerably certain scripts
when client-server latency is an issue. We should expect end users to
benefit from it too.

> I have some reservation as it will push all
> parameters in the query string which may not be the desired
> behaviour.

I don't follow. For me the change discussed here is about simplifying
the syntax when there is no out-of-query $N-style parameters, it does
not change anything for queries that actually use them, nor does
it forbid a \bind without parameters.


Best regards,
-- 
Daniel Vérité 
https://postgresql.verite.pro/

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. psql: Fix assertion failures with pipeline mode

  2. psql: Rework TAP routine psql_fails_like() to define WAL sender context

  3. psql: Split extended query protocol meta-commands in --help=commands

  4. psql: Improve descriptions of \\flush[request] in --help

  5. psql: Fix incorrect status code returned by \getresults

  6. psql: Allow queries terminated by semicolons while in pipeline mode

  7. psql: Add \sendpipeline to send query buffers while in a pipeline

  8. psql: Fix memory leak with \gx used within a pipeline

  9. psql: Add pipeline status to prompt and some state variables

  10. Add more tests for utility commands in pipelines

  11. psql: Add support for pipelines

  12. Add braces for if block with large comment in psql's common.c