Horton

Re: Direct I/O

Andres Freund <andres@anarazel.de>

View thread View on postgr.es
From: Andres Freund <andres@anarazel.de>
To: Noah Misch <noah@leadboat.com>
Cc: Thomas Munro <thomas.munro@gmail.com>, Heikki Linnakangas <hlinnaka@iki.fi>, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>, Justin Pryzby <pryzby@telsasoft.com>, pgsql-hackers@postgresql.org
Date: 2023-04-11T17:53:35Z
Lists: pgsql-hackers 
Hi,

On 2023-04-09 16:40:54 -0700, Noah Misch wrote:
> On Sun, Apr 09, 2023 at 02:45:16PM -0700, Andres Freund wrote:
> > It's not *just* that scenario. With a few concurrent connections you can get
> > into problematic territory even with halfway reasonable shared buffers.
>
> I am not familiar with such cases.  You could get there with 64MB shared
> buffers and 256 simultaneous commits of new-refilenode-creating transactions,
> but I'd still file that under going out of one's way to use tiny shared
> buffers relative to the write activity.  What combination did you envision?

I'd not say it's common, but it's less crazy than running with 128kB of s_b...

There's also the issue that log_newpage_range() is used in number of places
where we could have a lot of pre-existing buffer pins. So pinning another 64
buffers could tip us over.

Greetings,

Andres Freund

Commits

  1. Rename hook functions for debug_io_direct to match variable name.

  2. Rename io_direct to debug_io_direct.

  3. Skip the 004_io_direct.pl test if a pre-flight check fails.

  4. Use higher wal_level for 004_io_direct.pl.

  5. Skip \password TAP test on old IPC::Run versions

  6. Add io_direct setting (developer-only).

  7. Introduce PG_IO_ALIGN_SIZE and align all I/O buffers.

  8. Add palloc_aligned() to allow aligned memory allocations

  9. initdb: When running CREATE DATABASE, use STRATEGY = WAL_COPY.