Horton
Menu

Re: WIP: WAL prefetch (another approach)

David Steele <david@pgmasters.net>

View thread
From: David Steele <david@pgmasters.net>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Andres Freund <andres@anarazel.de>, Alvaro Herrera <alvherre@2ndquadrant.com>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2020-04-08T12:27:39Z
Lists: pgsql-hackers 
On 4/8/20 8:12 AM, Thomas Munro wrote:
> 
> Ok, so the following parts of this work have been committed:
> 
> b09ff536:  Simplify the effective_io_concurrency setting.
> fc34b0d9:  Introduce a maintenance_io_concurrency setting.
> 3985b600:  Support PrefetchBuffer() in recovery.
> d140f2f3:  Rationalize GetWalRcv{Write,Flush}RecPtr().
> 
> However, I didn't want to push the main patch into the tree at
> (literally) the last minute after doing such much work on it in the
> last few days, without more review from recovery code experts and some
> independent testing.  

I definitely think that was the right call.

> Judging by the comments made in this thread and
> elsewhere, I think the feature is in demand so I hope there is a way
> we could get it into 13 in the next couple of days, but I totally
> accept the release management team's prerogative on that.

That's up to the RMT, of course, but we did already have an extra week. 
Might be best to just get this in at the beginning of the PG14 cycle. 
FWIW, I do think the feature is really valuable.

Looks like you'll need to rebase, so I'll move this to the next CF in 
WoA state.

Regards,
-- 
-David
david@pgmasters.net

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Fix recovery_prefetch docs.

  2. Prefetch data referenced by the WAL, take II.

  3. Add circular WAL decoding buffer, take II.

  4. Fix generation of ./INSTALL for the distribution tarball

  5. Revert recovery prefetching feature.

  6. Sync guc.c and postgresql.conf.sample with the SGML docs.

  7. Add information of total data processed to replication slot stats.

  8. Doc: Review for "Optionally prefetch referenced data in recovery."

  9. Add circular WAL decoding buffer.

  10. Optionally prefetch referenced data in recovery.

  11. Remove read_page callback from XLogReader.

  12. Provide ReadRecentBuffer() to re-pin buffers by ID.

  13. Provide recovery_init_sync_method=syncfs.

  14. Mark factorial operator, and postfix operators in general, as deprecated.

  15. Rationalize GetWalRcv{Write,Flush}RecPtr().

  16. Support PrefetchBuffer() in recovery.

  17. Prevent hard failures of standbys caused by recycled WAL segments