Re: macOS prefetching support

Peter Eisentraut <peter@eisentraut.org>

From: Peter Eisentraut <peter@eisentraut.org>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2024-08-23T12:28:10Z
Lists: pgsql-hackers

Attachments

On 18.08.24 15:35, Peter Eisentraut wrote:
> On 17.08.24 00:01, Thomas Munro wrote:
>> Hmm.  fd.h and fd.c test for F_NOCACHE, which is pretty closely
>> related.  Now I'm wondering why we actually need this in
>> pg_config_manual.h at all.  Who would turn it off at compile time, and
>> why would they not be satisfied with setting relevant GUCs to 0?  Can
>> we just teach fd.h to define USE_PREFETCH if
>> defined(POSIX_FADV_WILLNEED) || defined(F_RDADVISE)?
> 
> I thought USE_PREFETCH existed so that we don't have the run-time 
> overhead for all the bookkeeping code if we don't have any OS-level 
> prefetch support at the end.  But it looks like most of that bookkeeping 
> code is skipped anyway if the *_io_concurrency settings are at 0.  So 
> yes, getting rid of USE_PREFETCH globally would be useful.
> 
>> (I have also thought multiple times about removing the configure
>> probes for F_FULLFSYNC, and just doing #ifdef.  Oh, that's in my patch
>> for CF #4453.)
> 
> Understandable, but we should be careful here that we don't create 
> setups that can cause bugs like 
> <https://www.postgresql.org/message-id/48da4a1f-ccd9-4988-9622-24f37b1de2b4@eisentraut.org>.
> 
>> I think that's fine.  I don't really like the word "prefetch", could
>> mean many different things.  What about "requires OS support for
>> issuing read-ahead advice", which uses a word that appears in both of
>> those interfaces?
> 
> I like that term.

Here is another patch, with the documentation wording adjusted like 
this, and a bit of other tidying.

I opted against pursuing some of the other refactoring ideas as part of 
this.  Removing USE_PREFETCH seems worthwhile, but has some open 
questions.  For example, pg_prewarm has a prefetch mode, which currently 
errors if there is no prefetch support.  So we'd still require some 
knowledge outside of fd.c, unless we want to change that behavior.  The 
idea of creating a src/port/posix_fadvise.c also got a bit too 
complicated in terms of how to weave that into configure and meson. 
There are apparently various old problems, like old Linux systems with 
incompatible declarations, or something like that, and then the special 
casing of Solaris (which isn't even in meson.build).  If we could get 
rid of some of that, then it would be easier to add new behavior there, 
otherwise it's just likely to break things.  So I'm leaving these as 
separate projects.

In terms of $subject, this patch seems sufficient for now.

Commits

  1. Standardize "read-ahead advice" terminology.

  2. Fixup for prefetching support on macOS

  3. Add prefetching support on macOS