Horton

Re: Spread checkpoint sync

Jeff Janes <jeff.janes@gmail.com>

View thread
From: Jeff Janes <jeff.janes@gmail.com>
To: Greg Smith <greg@2ndquadrant.com>
Cc: Robert Haas <robertmhaas@gmail.com>, Ron Mayer <rm_pg@cheapcomplexdevices.com>, Josh Berkus <josh@agliodbs.com>, pgsql-hackers@postgresql.org
Date: 2011-01-17T16:40:31Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add new buffers_backend_fsync field to pg_stat_bgwriter.

On Sun, Jan 16, 2011 at 7:13 PM, Greg Smith <greg@2ndquadrant.com> wrote:
> I have finished a first run of benchmarking the current 9.1 code at various
> sizes.  See http://www.2ndquadrant.us/pgbench-results/index.htm for many
> details.  The interesting stuff is in Test Set 3, near the bottom.  That's
> the first one that includes buffer_backend_fsync data.  This iall on ext3 so
> far, but is using a newer 2.6.32 kernel, the one from Ubuntu 10.04.
>
> The results are classic Linux in 2010:  latency pauses from checkpoint sync
> will easily leave the system at a dead halt for a minute, with the worst one
> observed this time dropping still for 108 seconds.  That one is weird, but
> these two are completely averge cases:
>
> http://www.2ndquadrant.us/pgbench-results/210/index.html
> http://www.2ndquadrant.us/pgbench-results/215/index.html
>
> I think a helpful next step here would be to put Robert's fsync compaction
> patch into here and see if that helps.  There are enough backend syncs
> showing up in the difficult workloads (scale>=1000, clients >=32) that its
> impact should be obvious.

Have you ever tested Robert's other idea of having a metronome process
do a periodic fsync on a dummy file which is located on the same ext3fs
as the table files?  I think that that would be interesting to see.

Cheers,

Jeff