Horton
Menu

Re: Add Pipelining support in psql

Kirill Reshke <reshkekirill@gmail.com>

View thread
From: Kirill Reshke <reshkekirill@gmail.com>
To: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Date: 2024-11-27T10:46:36Z
Lists: pgsql-hackers 
On Wed, 27 Nov 2024 at 14:50, Anthonin Bonnefoy
<anthonin.bonnefoy@datadoghq.com> wrote:
>
> Hi,
>
> With \bind, \parse, \bind_named and \close, it is possible to issue
> queries from psql using the extended protocol. However, it wasn't
> possible to send those queries using pipelining and the only way to
> test pipelined queries was through pgbench's tap tests.

Hello, good concept. Our connection pooler testing will greatly
benefit from this feature. At the moment, our tests are golang-based
and build pipelines in extended proto and verify the outcome.
Regression tests based on SQL will be far more thorough and organic.

> The attached patch adds pipelining support to psql with 3 new
> meta-commands, mirroring what's already done in pgbench:
> - \startpipeline starts a new pipeline. All extended queries will be
> queued until the end of the pipeline is reached.
> - \endpipeline ends an ongoing pipeline. All queued commands will be
> sent to the server and all responses will be processed by the psql.
> - \syncpipeline queue a synchronisation point without flushing the
> commands to the server

I'm very doubtful about the \syncpipeline . Maybe we should instead
support \sync meta-command in psql? This will be a useful contribution
itself.

> Those meta-commands will allow testing pipelined query behaviour using
> psql regression tests.
>
> Regards,
> Anthonin

I haven't looked into the patch in detail yet.

-- 
Best regards,
Kirill Reshke

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. psql: Fix assertion failures with pipeline mode

  2. psql: Rework TAP routine psql_fails_like() to define WAL sender context

  3. psql: Split extended query protocol meta-commands in --help=commands

  4. psql: Improve descriptions of \\flush[request] in --help

  5. psql: Fix incorrect status code returned by \getresults

  6. psql: Allow queries terminated by semicolons while in pipeline mode

  7. psql: Add \sendpipeline to send query buffers while in a pipeline

  8. psql: Fix memory leak with \gx used within a pipeline

  9. psql: Add pipeline status to prompt and some state variables

  10. Add more tests for utility commands in pipelines

  11. psql: Add support for pipelines

  12. Add braces for if block with large comment in psql's common.c