Horton

Re: Improve WALRead() to suck data directly from WAL buffers when possible

Jeff Davis <pgsql@j-davis.com>

View thread
From: Jeff Davis <pgsql@j-davis.com>
To: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Cc: Andres Freund <andres@anarazel.de>, Dilip Kumar <dilipbalaut@gmail.com>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, pgsql-hackers@lists.postgresql.org, Nathan Bossart <nathandbossart@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>
Date: 2023-11-04T21:27:39Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add XLogCtl->logInsertResult

  2. Add assert to WALReadFromBuffers().

  3. Read WAL directly from WAL buffers.

  4. Additional write barrier in AdvanceXLInsertBuffer().

  5. Use 64-bit atomics for xlblocks array elements.

  6. Don't trust unvalidated xl_tot_len.

On Sat, 2023-11-04 at 20:55 +0530, Bharath Rupireddy wrote:
> +		XLogRecPtr	EndPtr =
> pg_atomic_read_u64(&XLogCtl->xlblocks[curridx]);
> +
> +		/*
> +		 * xlblocks value can be InvalidXLogRecPtr before
> the new WAL buffer
> +		 * page gets initialized in AdvanceXLInsertBuffer.
> In such a case
> +		 * re-read the xlblocks value under the lock to
> ensure the correct
> +		 * value is read.
> +		 */
> +		if (unlikely(XLogRecPtrIsInvalid(EndPtr)))
> +		{
> +			LWLockAcquire(WALBufMappingLock,
> LW_EXCLUSIVE);
> +			EndPtr = pg_atomic_read_u64(&XLogCtl-
> >xlblocks[curridx]);
> +			LWLockRelease(WALBufMappingLock);
> +		}
> +
> +		Assert(!XLogRecPtrIsInvalid(EndPtr));

Can that really happen? If the EndPtr is invalid, that means the page
is in the process of being cleared, so the contents of the page are
undefined at that time, right?

Regards,
	Jeff Davis