Horton

Re: Spread checkpoint sync

Bruce Momjian <bruce@momjian.us>

View thread
From: Bruce Momjian <bruce@momjian.us>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Robert Haas <robertmhaas@gmail.com>, Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>, Itagaki Takahiro <itagaki.takahiro@gmail.com>, Greg Smith <greg@2ndquadrant.com>, Ron Mayer <rm_pg@cheapcomplexdevices.com>, Josh Berkus <josh@agliodbs.com>, pgsql-hackers@postgresql.org
Date: 2011-02-01T18:48:42Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add new buffers_backend_fsync field to pg_stat_bgwriter.

Tom Lane wrote:
> Bruce Momjian <bruce@momjian.us> writes:
> > My trivial idea was:  let's assume we checkpoint every 10 minutes, and
> > it takes 5 minutes for us to write the data to the kernel.   If no one
> > else is writing to those files, we can safely wait maybe 5 more minutes
> > before issuing the fsync.  If, however, hundreds of writes are coming in
> > for the same files in those final 5 minutes, we should fsync right away.
> 
> Huh?  I would surely hope we could assume that nobody but Postgres is
> writing the database files?  Or are you considering that the bgwriter
> doesn't know exactly what the backends are doing?  That's true, but
> I still maintain that we should design the bgwriter's behavior on the
> assumption that writes from backends are negligible.  Certainly the
> backends aren't issuing fsyncs.

Right, no one else is writing but us.  When I said "no one else" I meant
no other bgwrites writes are going to the files we wrote as part of the
checkpoint, but have not fsync'ed yet.  I assume we have two write
streams --- the checkpoint writes, which we know at the start of the
checkpoint, and the bgwriter writes that are happening in an
unpredictable way based on database activity.

-- 
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + It's impossible for everything to be true. +