Re: WIP: WAL prefetch (another approach)

Justin Pryzby <pryzby@telsasoft.com>

From: Justin Pryzby <pryzby@telsasoft.com>

To: Thomas Munro <thomas.munro@gmail.com>

Cc: Tomas Vondra <tomas.vondra@enterprisedb.com>, Stephen Frost <sfrost@snowman.net>, Andres Freund <andres@anarazel.de>, Jakub Wartak <Jakub.Wartak@tomtom.com>, Alvaro Herrera <alvherre@2ndquadrant.com>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, Dmitry Dolgov <9erthalion6@gmail.com>, David Steele <david@pgmasters.net>, pgsql-hackers@postgresql.org

Date: 2022-04-07T12:55:55Z

Lists: pgsql-hackers

The docs seem to be wrong about the default.

+        are not yet in the buffer pool, during recovery.  Valid values are
+        <literal>off</literal> (the default), <literal>on</literal> and
+        <literal>try</literal>.  The setting <literal>try</literal> enables

+   concurrency and distance, respectively.  By default, it is set to
+   <literal>try</literal>, which enabled the feature on systems where
+   <function>posix_fadvise</function> is available.

Should say "which enables".

+       {
+               {"recovery_prefetch", PGC_SIGHUP, WAL_RECOVERY,
+                       gettext_noop("Prefetch referenced blocks during recovery"),
+                       gettext_noop("Look ahead in the WAL to find references to uncached data.")
+               },
+               &recovery_prefetch,
+               RECOVERY_PREFETCH_TRY, recovery_prefetch_options,
+               check_recovery_prefetch, assign_recovery_prefetch, NULL
+       },

Curiously, I reported a similar issue last year.

On Thu, Apr 08, 2021 at 10:37:04PM -0500, Justin Pryzby wrote:
> --- a/doc/src/sgml/wal.sgml
> +++ b/doc/src/sgml/wal.sgml
> @@ -816,9 +816,7 @@
>     prefetching mechanism is most likely to be effective on systems
>     with <varname>full_page_writes</varname> set to
>     <varname>off</varname> (where that is safe), and where the working
> -   set is larger than RAM.  By default, prefetching in recovery is enabled
> -   on operating systems that have <function>posix_fadvise</function>
> -   support.
> +   set is larger than RAM.  By default, prefetching in recovery is disabled.
>    </para>
>   </sect1>

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Fix recovery_prefetch docs.
- dafae9707ab7 15.0 landed
Prefetch data referenced by the WAL, take II.
- 5dc0418fab28 15.0 landed
Add circular WAL decoding buffer, take II.
- 3f1ce973467a 15.0 landed
Fix generation of ./INSTALL for the distribution tarball
- 45aa88fe1d40 14.0 cited
Revert recovery prefetching feature.
- c2dc19342e05 14.0 landed
Sync guc.c and postgresql.conf.sample with the SGML docs.
- a55a98477b69 14.0 cited
Add information of total data processed to replication slot stats.
- f5fc2f5b23d1 14.0 cited
Doc: Review for "Optionally prefetch referenced data in recovery."
- dc88460c24ed 14.0 landed
Add circular WAL decoding buffer.
- f003d9f8721b 14.0 landed
Optionally prefetch referenced data in recovery.
- 1d257577e08d 14.0 landed
Remove read_page callback from XLogReader.
- 323cbe7c7ddc 14.0 cited
Provide ReadRecentBuffer() to re-pin buffers by ID.
- 2f27f8c51149 14.0 landed
Provide recovery_init_sync_method=syncfs.
- 61752afb2640 14.0 cited
Mark factorial operator, and postfix operators in general, as deprecated.
- 6ca547cf75ef 14.0 cited
Rationalize GetWalRcv{Write,Flush}RecPtr().
- d140f2f3e225 13.0 landed
Support PrefetchBuffer() in recovery.
- 3985b600f57d 13.0 landed
Prevent hard failures of standbys caused by recycled WAL segments
- 70b4f82a4b5c 11.0 cited