Re: Trying out read streams in pgvector (an extension)

Nazir Bilal Yavuz <byavuz81@gmail.com>

From: Nazir Bilal Yavuz <byavuz81@gmail.com>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Melanie Plageman <melanieplageman@gmail.com>, "Jonathan S. Katz" <jkatz@postgresql.org>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2025-11-12T08:04:28Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. Add read_stream_{pause,resume}()

Hi,

Thank you for working on this!

On Wed, 12 Nov 2025 at 07:12, Thomas Munro <thomas.munro@gmail.com> wrote:
>
> On Wed, Nov 12, 2025 at 12:19 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> > On Wed, Nov 12, 2025 at 11:52 AM Melanie Plageman
> > <melanieplageman@gmail.com> wrote:
> > > If we are worried about regressing other extensions using
> > > read_stream_reset(), we could make the read stream reset which
> > > preserves the distance a different function in backbranches.
>
> Here is a draft patch like that, that tries to be as small as
> possible.  Trying out the name read_stream_resume().

I liked the idea of having a different function named
read_stream_resume for this purpose.

0001 looks good to me.

0002:

+        /* End-of-stream. */
+        buf = read_stream_next_buffer(stream, NULL);
+        Assert(buf == InvalidBuffer);
+        buf = read_stream_next_buffer(stream, NULL);
+        Assert(buf == InvalidBuffer);

I noticed there are two 'read_stream_next_buffer()' and
'InvalidBuffer' checks. Does having both provide any additional
validation? I tried removing one of them, and the test still passed.

Also, there is one thing I wanted to clarify about the
'read_stream_resume()'. If 'read_stream_next_buffer()' returns an
'InvalidBuffer', then we can use 'read_stream_resume()' alone because
we know that we already consumed all buffers in the stream. For the
rest, we need to use 'read_stream_resume()' together with the
'read_stream_reset()', right?

-- 
Regards,
Nazir Bilal Yavuz
Microsoft