Re: Backup throttling

On 8/27/13 7:58 AM, Robert Haas wrote:
>  We have a *general* need to be able to throttle server-side resource
> utilization, particularly I/O.  This is a problem not only for
> pg_basebackup, but for COPY, CLUSTER, VACUUM, and even things like
> UPDATE.  Of all of those, the only one for which we currently have any
> kind of a solution is VACUUM.

I didn't mention it specifically, but I always presumed that the "Cost 
limited statements RFC" proposal I floated: 
http://www.postgresql.org/message-id/519EA5FF.5040606@2ndQuadrant.com 
(and am still working on) would handle the base backup case too. 
pg_basebackup is just another client.  Some sort of purpose made 
solution for pg_basebackup alone may be useful, but I'd be shocked if 
the sort of general mechanism I described there wasn't good enough to 
handle many of the backup limiting cases too.

Also, once that and the block write counters I also sent an RFC out for 
are in place, I have a plan for adding a server-wide throttling 
mechanism.  I want to extract a cluster wide read and write rate and put 
a cluster wide limit on that whole thing.  It seemed too hard to jump 
into without these other two pieces of plumbing in place first.

-- 
Greg Smith   2ndQuadrant US    greg@2ndQuadrant.com   Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com