Horton

Re: Direct I/O

John Naylor <john.naylor@enterprisedb.com>

View thread View on postgr.es
From: John Naylor <john.naylor@enterprisedb.com>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2022-11-10T07:26:20Z
Lists: pgsql-hackers 
On Tue, Nov 1, 2022 at 2:37 PM Thomas Munro <thomas.munro@gmail.com> wrote:

> Memory alignment patches:
>
> Direct I/O generally needs to be done to/from VM page-aligned
> addresses, but only "standard" 4KB pages, even when larger VM pages
> are in use (if there is an exotic system where that isn't true, it
> won't work).  We need to deal with buffers on the stack, the heap and
> in shmem.  For the stack, see patch 0001.  For the heap and shared
> memory, see patch 0002, but David Rowley is going to propose that part
> separately, as MemoryContext API adjustments are a specialised enough
> topic to deserve another thread; here I include a copy as a
> dependency.  The main direct I/O patch is 0003.

One thing to note: Currently, a request to aset above 8kB must go into a
dedicated block. Not sure if it's a coincidence that that matches the
default PG page size, but if allocating pages on the heap is hot enough,
maybe we should consider raising that limit. Although then, aligned-to-4kB
requests would result in 16kB chunks requested unless a different allocator
was used.

--
John Naylor
EDB: http://www.enterprisedb.com

Commits

  1. Rename hook functions for debug_io_direct to match variable name.

  2. Rename io_direct to debug_io_direct.

  3. Skip the 004_io_direct.pl test if a pre-flight check fails.

  4. Use higher wal_level for 004_io_direct.pl.

  5. Skip \password TAP test on old IPC::Run versions

  6. Add io_direct setting (developer-only).

  7. Introduce PG_IO_ALIGN_SIZE and align all I/O buffers.

  8. Add palloc_aligned() to allow aligned memory allocations

  9. initdb: When running CREATE DATABASE, use STRATEGY = WAL_COPY.