Horton

Re: our buffer replacement strategy is kind of lame

Simon Riggs <simon@2ndquadrant.com>

View thread
From: Simon Riggs <simon@2ndQuadrant.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Greg Stark <stark@mit.edu>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2011-08-14T10:57:22Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Split work of bgwriter between 2 processes: bgwriter and checkpointer.

On Sat, Aug 13, 2011 at 11:14 PM, Robert Haas <robertmhaas@gmail.com> wrote:
> On Sat, Aug 13, 2011 at 4:40 PM, Greg Stark <stark@mit.edu> wrote:
>> On Sat, Aug 13, 2011 at 8:52 PM, Robert Haas <robertmhaas@gmail.com> wrote:
>>> and possibly we ought to put them all in a
>>> linked list so that the next guy who needs a buffer can just pop one
>>
>> The whole point of the clock sweep algorithm is to approximate an LRU
>> without needing to maintain a linked list. The problem with a linked
>> list is that you need to serialize access to it so every time you
>> reference a buffer you need to wait on a lock for the list so you can
>> move that buffer around in the list.
>
> Right, but the same doesn't apply to what I'm talking about.  You just
> put each guy into the linked list when his reference count goes down
> to 0.  When you want to allocate a buffer, you pop somebody off.  If
> his reference count is still 0, you forget about him and pop the next
> guy, and keep going until you find a suitable victim.
>
> The problem with just running the clock sweep every time you need a
> victim buffer is that you may have to pass over a large number of
> unevictable buffers before you find someone you can actually kick out.
>  That's work that should really be getting done in the background, not
> at the absolute last moment when we discover, hey, we need a buffer.

I think Greg is right here. Those suggested changes just bring back the LRU.

If you keep a separate list then you need to serialize access to it.

usage_count == 0 and "unevictable" are dynamic states, so if the
bgwriter sees those states they can change before anyone uses the
buffer.

The only state which is unchangeable is a recently filled block, such
as from a COPY, which is why I originally suggested a
filled-block-list - though I don't think we need it now because of the
buffer cycle strategy.

-- 
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services