Horton

Re: Improve WALRead() to suck data directly from WAL buffers when possible

Andres Freund <andres@anarazel.de>

View thread
From: Andres Freund <andres@anarazel.de>
To: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Cc: Jeff Davis <pgsql@j-davis.com>, Dilip Kumar <dilipbalaut@gmail.com>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, pgsql-hackers@lists.postgresql.org, Nathan Bossart <nathandbossart@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>
Date: 2024-01-22T20:12:18Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add XLogCtl->logInsertResult

  2. Add assert to WALReadFromBuffers().

  3. Read WAL directly from WAL buffers.

  4. Additional write barrier in AdvanceXLInsertBuffer().

  5. Use 64-bit atomics for xlblocks array elements.

  6. Don't trust unvalidated xl_tot_len.

Hi,

On 2024-01-10 19:59:29 +0530, Bharath Rupireddy wrote:
> +		/*
> +		 * Typically, we must not read a WAL buffer page that just got
> +		 * initialized. Because we waited enough for the in-progress WAL
> +		 * insertions to finish above. However, there can exist a slight
> +		 * window after the above wait finishes in which the read buffer page
> +		 * can get replaced especially under high WAL generation rates. After
> +		 * all, we are reading from WAL buffers without any locks here. So,
> +		 * let's not count such a page in.
> +		 */
> +		phdr = (XLogPageHeader) page;
> +		if (!(phdr->xlp_magic == XLOG_PAGE_MAGIC &&
> +			  phdr->xlp_pageaddr == (ptr - (ptr % XLOG_BLCKSZ)) &&
> +			  phdr->xlp_tli == tli))
> +			break;

I still think that anything that requires such checks shouldn't be
merged. It's completely bogus to check page contents for validity when we
should have metadata telling us which range of the buffers is valid and which
not.

Greetings,

Andres Freund