Re: tuning questions

Jack,

> latest changes:
> shared_buffers = 35642

This is fine, it's about 14% of available RAM.  Though the way you calculated 
it still confuses me.   It's not complicated; it should be between 6% and 15% 
of available RAM; since you're doing a data-transformation DB, yours should 
be toward the high end. 

> max_fsm_relations = 1000
> max_fsm_pages = 10000

You want to raise this a whole lot if your data transformations involve large 
delete or update batches.    I'd suggest running "vacuum analyze verbose" 
between steps to see how many dead pages you're accumulating.

> wal_buffers = 64
> sort_mem = 32768
> vacuum_mem = 32768
> effective_cache_size = 10000

This is way the heck too low.  it's supposed to be the size of all available 
RAM; I'd set it to 2GB*65% as a start.

> IO is active, but hardly saturated. CPU load is hefty though, load
> average is at 4 now.

Unless you're doing huge statistical aggregates (like radar charts), or heavy 
numerical calculations-by-query, high CPU and idle I/O usually indicates a 
really bad query, like badly mismatched data types on a join or unconstrained 
joins or  overblown formatting-by-query.

-- 
-Josh Berkus
 Aglio Database Solutions
 San Francisco