Re: Direct I/O

On 2023-04-12 We 12:38, Dagfinn Ilmari Mannsåker wrote:
> Andrew Dunstan<andrew@dunslane.net>  writes:
>
>> On 2023-04-12 We 10:23, Dagfinn Ilmari Mannsåker wrote:
>>> Andrew Dunstan<andrew@dunslane.net>   writes:
>>>
>>>> On 2023-04-12 We 01:48, Thomas Munro wrote:
>>>>> On Wed, Apr 12, 2023 at 3:04 PM Thomas Munro<thomas.munro@gmail.com>    wrote:
>>>>>> On Wed, Apr 12, 2023 at 2:56 PM Christoph Berg<myon@debian.org>    wrote:
>>>>>>> I'm hitting a panic in t_004_io_direct. The build is running on
>>>>>>> overlayfs on tmpfs/ext4 (upper/lower) which is probably a weird
>>>>>>> combination but has worked well for building everything over the last
>>>>>>> decade. On Debian unstable:
>>>>>>>
>>>>>>> PANIC:  could not open file "pg_wal/000000010000000000000001": Invalid argument
>>>>>> ... I have a new idea:  perhaps it is possible to try
>>>>>> to open a file with O_DIRECT from perl, and if it fails like that,
>>>>>> skip the test.  Looking into that now.
>>>>> I think I have that working OK.  Any Perl hackers want to comment on
>>>>> my use of IO::File (copied from examples on the internet that showed
>>>>> how to use O_DIRECT)?  I am not much of a perl hacker but according to
>>>>> my package manager, IO/File.pm came with perl itself.  And the Fcntl
>>>>> eval trick that I copied from File::stat, and the perl-critic
>>>>> suppression that requires?
>>>> I think you can probably replace a lot of the magic here by simply saying
>>>>
>>>>
>>>> if (Fcntl->can("O_DIRECT")) ...
>>> Fcntl->can() is true for all constants that Fcntl knows about, whether
>>> or not they are defined for your OS. `defined &O_DIRECT` is the simplest
>>> check, see my other reply to Thomas.
>>>
>>>
>> My understanding was that Fcntl only exported constants known to the
>> OS. That's certainly what its docco suggests, e.g.:
>>
>>      By default your system's F_* and O_* constants (eg, F_DUPFD and
>> O_CREAT)
>>      and the FD_CLOEXEC constant are exported into your namespace.
> It's a bit more magical than that (this is Perl after all).  They are
> all exported (which implicitly creates stubs visible to `->can()`,
> similarly to forward declarations like `sub O_FOO;`), but only the
> defined ones (`#ifdef O_FOO` is true) are defined (`defined &O_FOO` is
> true).  The rest fall through to an AUTOLOAD¹ function that throws an
> exception for undefined ones.
>
> Here's an example (Fcntl knows O_RAW, but Linux does not define it):
>
>      $ perl -E '
>          use strict; use Fcntl;
>          say "can", main->can("O_RAW") ? "" : "not";
>          say defined &O_RAW ? "" : "not ", "defined";
>          say O_RAW;'
>      can
>      not defined
>      Your vendor has not defined Fcntl macro O_RAW, used at -e line 4
>
> While O_DIRECT is defined:
>
>      $ perl -E '
>          use strict; use Fcntl;
>          say "can", main->can("O_DIRECT") ? "" : "not";
>          say defined &O_DIRECT ? "" : "not ", "defined";
>          say O_DIRECT;'
>      can
>      defined
>      16384
>
> And O_FOO is unknown to Fcntl (the parens on `O_FOO()q are to make it
> not a bareword, which would be a compile error under `use strict;` when
> the sub doesn't exist at all):
>
>      $ perl -E '
>          use strict; use Fcntl;
>          say "can", main->can("O_FOO") ? "" : "not";
>          say defined &O_FOO ? "" : "not ", "defined";
>          say O_FOO();'
>      cannot
>      not defined
>      Undefined subroutine &main::O_FOO called at -e line 4.
>
>

*grumble* a bit too magical for my taste. Thanks for the correction.


cheers


andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com