Re: PATCH: Split stats file per database WAS: autovacuum stress-testing our system

Tomas Vondra <tv@fuzzy.cz>

From: Tomas Vondra <tv@fuzzy.cz>
To: <pgsql-hackers@postgresql.org>
Date: 2013-02-06T16:26:29Z
Lists: pgsql-hackers
Dne 06.02.2013 16:53, Alvaro Herrera napsal:
> Tom Lane escribió:
>> Pavel Stehule <pavel.stehule@gmail.com> writes:
>> >> Nice. Another interesting numbers would be device utilization, 
>> average
>> >> I/O speed and required space (which should be ~2x the pgstat.stat 
>> size
>> >> without the patch).
>>
>> > this point is important - with large warehouse with lot of 
>> databases
>> > and tables you have move stat file to some ramdisc - without it 
>> you
>> > lost lot of IO capacity - and it is very important if you need 
>> only
>> > half sized ramdisc
>>
>> [ blink... ]  I confess I'd not been paying close attention to this
>> thread, but if that's true I'd say the patch is DOA.  Why should we
>> accept 2x bloat in the already-far-too-large stats file?  I thought
>> the idea was just to split up the existing data into multiple files.
>
> I think they are saying just the opposite: maximum disk space
> utilization is now half of the unpatched code.  This is because when 
> we
> need to write the temporary file to rename on top of the other one, 
> the
> temporary file is not of the size of the complete pgstat data 
> collation,
> but just that for the requested database.

Exactly. And I suspect the current (unpatched) code ofter requires more 
than
twice the space because of open file descriptors to already deleted 
files.

Tomas


Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. Split pgstat file in smaller pieces