Re: XLogInsert scaling, revisited

On 01.07.2013 16:40, Andres Freund wrote:
> On 2013-06-26 18:52:30 +0300, Heikki Linnakangas wrote:
>>> * Could you document the way slots prevent checkpoints from occurring
>>>    when XLogInsert rechecks for full page writes? I think it's correct -
>>>    but not very obvious on a glance.
>>
>> There's this in the comment near the top of the file:
>>
>>   * To update RedoRecPtr or fullPageWrites, one has to make sure that all
>>   * subsequent inserters see the new value. This is done by reserving all the
>>   * insertion slots before changing the value. XLogInsert reads RedoRecPtr
>> and
>>   * fullPageWrites after grabbing a slot, so by holding all the slots
>>   * simultaneously, you can ensure that all subsequent inserts see the new
>>   * value.  Those fields change very seldom, so we prefer to be fast and
>>   * non-contended when they need to be read, and slow when they're changed.
>>
>> Does that explain it well enough? XLogInsert holds onto a slot while it
>> rechecks for full page writes.
>
> I am a bit worried about that logic. We're basically reverting to the
> old logic whe xlog writing is an exlusive affair. We will have to wait
> for all the other queued inserters before we're finished. I am afraid
> that that will show up latencywise.

A single stall of the xlog-insertion "pipeline" at a checkpoint is 
hardly going to be a problem. I wish PostgreSQL was real-time enough for 
that to matter, but I think we're very very far from that.

- Heikki