Horton
Menu

Re: [EXTERNAL] Re: WIP: WAL prefetch (another approach)

Stephen Frost <sfrost@snowman.net>

View thread
From: Stephen Frost <sfrost@snowman.net>
To: Andres Freund <andres@anarazel.de>
Cc: Sait Talha Nisanci <Sait.Nisanci@microsoft.com>, Thomas Munro <thomas.munro@gmail.com>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, Dmitry Dolgov <9erthalion6@gmail.com>, David Steele <david@pgmasters.net>, Alvaro Herrera <alvherre@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2020-08-27T18:51:16Z
Lists: pgsql-hackers 
Greetings,

* Andres Freund (andres@anarazel.de) wrote:
> On August 27, 2020 11:26:42 AM PDT, Stephen Frost <sfrost@snowman.net> wrote:
> >Is WAL FPW compression enabled..?  I'm trying to figure out how, given
> >what's been shared here, that replaying 25GB of WAL is being helped out
> >by 2.5x thanks to prefetch in the SSD case.  That prefetch is hurting
> >in
> >the HDD case entirely makes sense to me- we're spending time reading
> >pages from the HDD, which is entirely pointless work given that we're
> >just going to write over those pages entirely with FPWs.
> 
> Hm? At least earlier versions didn't do prefetching for records with an fpw, and only for subsequent records affecting the same or if not in s_b anymore.

We don't actually read the page when we're replaying an FPW though..?
If we don't read it, and we entirely write the page from the FPW, how is
pre-fetching helping..?  I understood how it could be helpful for
filesystems which have a larger block size than ours (eg: zfs w/ 16kb
block sizes where the kernel needs to get the whole 16kb block when we
only write 8kb to it), but that's apparently not the case here.

So- what is it that pre-fetching is doing to result in such an
improvement?  Is there something lower level where the SSD physical
block size is coming into play, which is typically larger..?  I wouldn't
have thought so, but perhaps that's the case..

Thanks,

Stephen

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Fix recovery_prefetch docs.

  2. Prefetch data referenced by the WAL, take II.

  3. Add circular WAL decoding buffer, take II.

  4. Fix generation of ./INSTALL for the distribution tarball

  5. Revert recovery prefetching feature.

  6. Sync guc.c and postgresql.conf.sample with the SGML docs.

  7. Add information of total data processed to replication slot stats.

  8. Doc: Review for "Optionally prefetch referenced data in recovery."

  9. Add circular WAL decoding buffer.

  10. Optionally prefetch referenced data in recovery.

  11. Remove read_page callback from XLogReader.

  12. Provide ReadRecentBuffer() to re-pin buffers by ID.

  13. Provide recovery_init_sync_method=syncfs.

  14. Mark factorial operator, and postfix operators in general, as deprecated.

  15. Rationalize GetWalRcv{Write,Flush}RecPtr().

  16. Support PrefetchBuffer() in recovery.

  17. Prevent hard failures of standbys caused by recycled WAL segments