Horton

Re: Spread checkpoint sync

Martijn van Oosterhout <kleptog@svana.org>

View thread
From: Martijn van Oosterhout <kleptog@svana.org>
To: Greg Smith <greg@2ndquadrant.com>
Cc: Robert Haas <robertmhaas@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2010-11-21T22:19:30Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add new buffers_backend_fsync field to pg_stat_bgwriter.

On Sun, Nov 21, 2010 at 04:54:00PM -0500, Greg Smith wrote:
> Ultimately what I want to do here is some sort of smarter write-behind  
> sync operation, perhaps with a LRU on relations with pending fsync  
> requests.  The idea would be to sync relations that haven't been touched  
> in a while in advance of the checkpoint even.  I think that's similar to  
> the general idea Robert is suggesting here, to get some sync calls  
> flowing before all of the checkpoint writes have happened.  I think that  
> the final sync calls will need to get spread out regardless, and since  
> doing that requires a fairly small amount of code too that's why we  
> started with that.

For a similar problem we had (kernel buffering too much) we had success
using the fadvise and madvise WONTNEED syscalls to force the data to
exit the cache much sooner than it would otherwise. This was on Linux
and it had the side-effect that the data was deleted from the kernel
cache, which we wanted, but probably isn't appropriate here.

There is also sync_file_range, but that's linux specific, although
close to what you want I think. It would allow you to work with blocks
smaller than 1GB.

Have a nice day,
-- 
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> Patriotism is when love of your own people comes first; nationalism,
> when hate for people other than your own comes first. 
>                                       - Charles de Gaulle