Horton

Re: Direct I/O

Noah Misch <noah@leadboat.com>

View thread View on postgr.es
From: Noah Misch <noah@leadboat.com>
To: Andres Freund <andres@anarazel.de>
Cc: Thomas Munro <thomas.munro@gmail.com>, Heikki Linnakangas <hlinnaka@iki.fi>, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>, Justin Pryzby <pryzby@telsasoft.com>, pgsql-hackers@postgresql.org
Date: 2023-04-09T23:40:54Z
Lists: pgsql-hackers 
On Sun, Apr 09, 2023 at 02:45:16PM -0700, Andres Freund wrote:
> On 2023-04-08 21:29:54 -0700, Noah Misch wrote:
> > On Sat, Apr 08, 2023 at 11:08:16AM -0700, Andres Freund wrote:
> > > On 2023-04-07 23:04:08 -0700, Andres Freund wrote:
> > > > If you look at log_newpage_range(), it's not surprising that we get this error
> > > > - it pins up to 32 buffers at once.
> > > > 
> > > > Afaics log_newpage_range() originates in 9155580fd5fc, but this caller is from
> > > > c6b92041d385.
> > 
> > > > Do we care about fixing this in the backbranches? Probably not, given there
> > > > haven't been user complaints?
> > 
> > I would not.  This is only going to come up where the user goes out of the way
> > to use near-minimum shared_buffers.
> 
> It's not *just* that scenario. With a few concurrent connections you can get
> into problematic territory even with halfway reasonable shared buffers.

I am not familiar with such cases.  You could get there with 64MB shared
buffers and 256 simultaneous commits of new-refilenode-creating transactions,
but I'd still file that under going out of one's way to use tiny shared
buffers relative to the write activity.  What combination did you envision?

Commits

  1. Rename hook functions for debug_io_direct to match variable name.

  2. Rename io_direct to debug_io_direct.

  3. Skip the 004_io_direct.pl test if a pre-flight check fails.

  4. Use higher wal_level for 004_io_direct.pl.

  5. Skip \password TAP test on old IPC::Run versions

  6. Add io_direct setting (developer-only).

  7. Introduce PG_IO_ALIGN_SIZE and align all I/O buffers.

  8. Add palloc_aligned() to allow aligned memory allocations

  9. initdb: When running CREATE DATABASE, use STRATEGY = WAL_COPY.