Re: Direct I/O

Hi,

On 2022-11-02 09:44:30 +1300, Thomas Munro wrote:
> On Wed, Nov 2, 2022 at 2:33 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
> > On Tue, Nov 01, 2022 at 08:36:18PM +1300, Thomas Munro wrote:
> > >   io_data_direct = whether to use O_DIRECT for main data files
> > >   io_wal_direct = ... for WAL
> > >   io_wal_init_direct = ... for WAL-file initialisation
> >
> > You added 3 booleans, but I wonder if it's better to add a string GUC
> > which is parsed for comma separated strings.

In the past more complicated GUCs have not been well received, but it does
seem like a nice way to reduce the amount of redundant stuff.

Perhaps we could use the guc assignment hook to transform the input value into
a bitmask?


> > (By "better", I mean reducing the number of new GUCs - which is less
> > important for developer GUCs anyway.)

FWIW, if / once we get to actual AIO, at least some of these would stop being
developer-only GUCs. There's substantial performance benefits in using DIO
with AIO. Buffered IO requires the CPU to copy the data from the userspace
into the kernelspace. But DIO can use DMA for that, freeing the CPU to do more
useful work.  Buffered IO tops out much much earlier than AIO + DIO, and
unfortunately tops out at much lower speeds on server CPUs.


> > DIO is slower, but not so much that it can't run under CI.  I suggest to
> > add an 099 commit to enable the feature during development.
> 
> Good idea, will do.

Might be worth to additionally have a short tap test that does some basic
stuff with DIO and leave that enabled? I think it'd be good to have
check-world exercise DIO on dev machines, to reduce the likelihood of finding
problems only in CI, which is somewhat painful.


> > Note that this fails under linux with fsanitize=align:
> > ../src/backend/storage/file/buffile.c:117:17: runtime error: member access within misaligned address 0x561a4a8e40f8 for type 'struct BufFile', which requires 4096 byte alignment
> 
> Oh, so BufFile is palloc'd and contains one of these.  BufFile is not
> even using direct I/O, but by these rules it would need to be
> palloc_io_align'd.  I will think about what to do about that...

It might be worth having two different versions of the struct, so we don't
impose unnecessarily high alignment everywhere?

Greetings,

Andres Freund