Horton
Menu

Re: Add Pipelining support in psql

Michael Paquier <michael@paquier.xyz>

View thread
From: Michael Paquier <michael@paquier.xyz>
To: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Cc: Jelte Fennema-Nio <postgres@jeltef.nl>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Date: 2025-02-21T02:33:41Z
Lists: pgsql-hackers

Attachments

On Thu, Feb 20, 2025 at 10:29:33AM +0100, Anthonin Bonnefoy wrote:
> Ha yeah, I forgot about the inactive branches. I've added the new
> commands and fixed the behaviour.

And I did not notice that it was as simple as forcing the status in
the routines for the new meta-commands, as we do for the existing
ones.  Noted.

> This looks more like an issue on libpq's side as there's no way to
> reset or advance the errorReported from ExecQueryAndProcessResults
> (plus PQerrorMessage seems to ignore errorReported). I've added an
> additional test to track this behaviour for now as this would probably
> be better discussed in a dedicated thread.

I am not sure if we should change that, actually, as it does not feel
completely wrong to stack these errors.  That's a bit confusing,
sure.  Perhaps a new libpq API to retrieve stacked errors when we are
in pipeline mode would be more adapted?  The design would be
different.

Anyway, I've stared at the result processing code for a couple of
hours, and the branches we're taking for the pipeline modes seem to be
rather right the way you have implemented them.  The docs, comments
and tests needed quite a few tweaks and edits to be more consistent.
There were some grammar mistakes, some frenchisms.

I'm hoping that there won't be any issues, but let's be honest, I am
definitely sure there will be some more tuning required.  It comes
down to if we want this set of features, and I do to be able to expand
tests in core with the extended query protocol and pipelines, knowing
that there is an ask for out-of-core projects.  This one being
reachable with a COPY gave me a huge smile:
+message type 0x5a arrived from server while idle

So let's take one step here, I have applied the main patch.  I am
really excited by the possibilities all this stuff offers.

Attached are the remaining pieces, split here because they are
different bullet points:
- Tests for implicit transactions with various commands, with some
edits. 
- Prompt support, with more edits.

I'm putting these on standby for a few days, to let the buildfarm
digest the main change.
--
Michael

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. psql: Fix assertion failures with pipeline mode

  2. psql: Rework TAP routine psql_fails_like() to define WAL sender context

  3. psql: Split extended query protocol meta-commands in --help=commands

  4. psql: Improve descriptions of \\flush[request] in --help

  5. psql: Fix incorrect status code returned by \getresults

  6. psql: Allow queries terminated by semicolons while in pipeline mode

  7. psql: Add \sendpipeline to send query buffers while in a pipeline

  8. psql: Fix memory leak with \gx used within a pipeline

  9. psql: Add pipeline status to prompt and some state variables

  10. Add more tests for utility commands in pipelines

  11. psql: Add support for pipelines

  12. Add braces for if block with large comment in psql's common.c