Re: Improve WALRead() to suck data directly from WAL buffers when possible

Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>

From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>

To: Kuntal Ghosh <kuntalghosh.2007@gmail.com>

Cc: Nathan Bossart <nathandbossart@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, pgsql-hackers@lists.postgresql.org

Date: 2023-03-02T11:43:32Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Add XLogCtl->logInsertResult
- f3ff7bf83bce 17.0 cited
Add assert to WALReadFromBuffers().
- 9ecbf54075a9 17.0 landed
Read WAL directly from WAL buffers.
- 91f2cae7a4e6 17.0 landed
Additional write barrier in AdvanceXLInsertBuffer().
- 766571be1659 17.0 landed
Use 64-bit atomics for xlblocks array elements.
- c3a8e2a7cb16 17.0 landed
Don't trust unvalidated xl_tot_len.
- bae868caf222 17.0 cited

Attachments

v6-0001-Improve-WALRead-to-suck-data-directly-from-WAL-bu.patch (application/x-patch) patch v6-0001
v6-0002-Add-test-module-for-verifying-WAL-read-from-WAL-b.patch (application/x-patch) patch v6-0002

On Wed, Mar 1, 2023 at 2:39 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:
>
> Please find the attached v5 patch set for further review.

I simplified the code largely by moving the logic of reading the WAL
buffer page from a separate function to the main while loop. This
enabled me to get rid of XLogReadFromBuffersGuts() that v5 and other
previous patches have.

Please find the attached v6 patch set for further review. Meanwhile,
I'll continue to work on the review comment raised upthread -
https://www.postgresql.org/message-id/20230301041523.GA1453450%40nathanxps13.

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com