Horton

Re: Direct I/O

Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>

View thread View on postgr.es
From: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
To: Andrew Dunstan <andrew@dunslane.net>
Cc: Thomas Munro <thomas.munro@gmail.com>, Christoph Berg <myon@debian.org>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2023-04-12T14:23:58Z
Lists: pgsql-hackers 
Andrew Dunstan <andrew@dunslane.net> writes:

> On 2023-04-12 We 01:48, Thomas Munro wrote:
>> On Wed, Apr 12, 2023 at 3:04 PM Thomas Munro<thomas.munro@gmail.com>  wrote:
>>> On Wed, Apr 12, 2023 at 2:56 PM Christoph Berg<myon@debian.org>  wrote:
>>>> I'm hitting a panic in t_004_io_direct. The build is running on
>>>> overlayfs on tmpfs/ext4 (upper/lower) which is probably a weird
>>>> combination but has worked well for building everything over the last
>>>> decade. On Debian unstable:
>>>>
>>>> PANIC:  could not open file "pg_wal/000000010000000000000001": Invalid argument
>>> ... I have a new idea:  perhaps it is possible to try
>>> to open a file with O_DIRECT from perl, and if it fails like that,
>>> skip the test.  Looking into that now.
>> I think I have that working OK.  Any Perl hackers want to comment on
>> my use of IO::File (copied from examples on the internet that showed
>> how to use O_DIRECT)?  I am not much of a perl hacker but according to
>> my package manager, IO/File.pm came with perl itself.  And the Fcntl
>> eval trick that I copied from File::stat, and the perl-critic
>> suppression that requires?
>
>
> I think you can probably replace a lot of the magic here by simply saying
>
>
> if (Fcntl->can("O_DIRECT")) ...

Fcntl->can() is true for all constants that Fcntl knows about, whether
or not they are defined for your OS. `defined &O_DIRECT` is the simplest
check, see my other reply to Thomas.

> cheers
>
>
> andrew

- ilmari

Commits

  1. Rename hook functions for debug_io_direct to match variable name.

  2. Rename io_direct to debug_io_direct.

  3. Skip the 004_io_direct.pl test if a pre-flight check fails.

  4. Use higher wal_level for 004_io_direct.pl.

  5. Skip \password TAP test on old IPC::Run versions

  6. Add io_direct setting (developer-only).

  7. Introduce PG_IO_ALIGN_SIZE and align all I/O buffers.

  8. Add palloc_aligned() to allow aligned memory allocations

  9. initdb: When running CREATE DATABASE, use STRATEGY = WAL_COPY.