Re: WIP: WAL prefetch (another approach)

Thomas Munro <thomas.munro@gmail.com>

From: Thomas Munro <thomas.munro@gmail.com>

To: David Steele <david@pgmasters.net>

Cc: Andres Freund <andres@anarazel.de>, Alvaro Herrera <alvherre@2ndquadrant.com>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>

Date: 2020-04-08T21:55:25Z

Lists: pgsql-hackers

Attachments

v8-0001-Add-pg_atomic_unlocked_add_fetch_XXX.patch (text/x-patch) patch v8-0001
v8-0002-Allow-XLogReadRecord-to-be-non-blocking.patch (text/x-patch) patch v8-0002
v8-0003-Prefetch-referenced-blocks-during-recovery.patch (text/x-patch) patch v8-0003

On Thu, Apr 9, 2020 at 12:27 AM David Steele <david@pgmasters.net> wrote:
> On 4/8/20 8:12 AM, Thomas Munro wrote:
> > Judging by the comments made in this thread and
> > elsewhere, I think the feature is in demand so I hope there is a way
> > we could get it into 13 in the next couple of days, but I totally
> > accept the release management team's prerogative on that.
>
> That's up to the RMT, of course, but we did already have an extra week.
> Might be best to just get this in at the beginning of the PG14 cycle.
> FWIW, I do think the feature is really valuable.
>
> Looks like you'll need to rebase, so I'll move this to the next CF in
> WoA state.

Thanks.  Here's a rebase.

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Fix recovery_prefetch docs.
- dafae9707ab7 15.0 landed
Prefetch data referenced by the WAL, take II.
- 5dc0418fab28 15.0 landed
Add circular WAL decoding buffer, take II.
- 3f1ce973467a 15.0 landed
Fix generation of ./INSTALL for the distribution tarball
- 45aa88fe1d40 14.0 cited
Revert recovery prefetching feature.
- c2dc19342e05 14.0 landed
Sync guc.c and postgresql.conf.sample with the SGML docs.
- a55a98477b69 14.0 cited
Add information of total data processed to replication slot stats.
- f5fc2f5b23d1 14.0 cited
Doc: Review for "Optionally prefetch referenced data in recovery."
- dc88460c24ed 14.0 landed
Add circular WAL decoding buffer.
- f003d9f8721b 14.0 landed
Optionally prefetch referenced data in recovery.
- 1d257577e08d 14.0 landed
Remove read_page callback from XLogReader.
- 323cbe7c7ddc 14.0 cited
Provide ReadRecentBuffer() to re-pin buffers by ID.
- 2f27f8c51149 14.0 landed
Provide recovery_init_sync_method=syncfs.
- 61752afb2640 14.0 cited
Mark factorial operator, and postfix operators in general, as deprecated.
- 6ca547cf75ef 14.0 cited
Rationalize GetWalRcv{Write,Flush}RecPtr().
- d140f2f3e225 13.0 landed
Support PrefetchBuffer() in recovery.
- 3985b600f57d 13.0 landed
Prevent hard failures of standbys caused by recycled WAL segments
- 70b4f82a4b5c 11.0 cited