Horton

Thread

  1. Group Commit

    Simon Riggs <simon@2ndquadrant.com> — 2011-11-14T10:08:01Z

    Enclosed patch implements Group Commit and also powersave mode for WALWriter.

XLogFlush() waits for WALWriter to run XLogBackgroundFlush(), which
flushes WAL and then wakes waiters. Uses same concepts and similar
code to sync rep.

Purpose is to provide consistent WAL writes, even when WALInsertLock
contended. Currently no "off" option, thinking is that the overhead of
doing this is relatively low and so it can be "always on" - exactly as
it is for sync rep.

WALWriter now has variable wakeups, so wal_writer_delay is removed.
Commit_delay and Commit_siblings are now superfluous and are also removed.

Works, but needs discussion in some areas, docs and possibly tuning
first, so this is more of a quicky than a slow, comfortable patch.

-- 
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

  2. Re: Group Commit

    Josh Berkus <josh@agliodbs.com> — 2011-11-14T20:43:19Z

    > Purpose is to provide consistent WAL writes, even when WALInsertLock
> contended. Currently no "off" option, thinking is that the overhead of
> doing this is relatively low and so it can be "always on" - exactly as
> it is for sync rep.

Hmmm, have you had a chance to do any performance tests?

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

  3. Re: Group Commit

    Greg Smith <greg@2ndquadrant.com> — 2011-11-14T21:37:34Z

    On 11/14/2011 03:43 PM, Josh Berkus wrote:
>    
>> Purpose is to provide consistent WAL writes, even when WALInsertLock
>> contended. Currently no "off" option, thinking is that the overhead of
>> doing this is relatively low and so it can be "always on" - exactly as
>> it is for sync rep.
>>      
> Hmmm, have you had a chance to do any performance tests?
>    

I was planning to run some later this week, but someone else is welcome 
to take a shot at it.  The inspiration for this change was the 
performance scaling tests I did for sync rep last month.  Don't recall 
if I shared those with this list yet; I've attached the fun graph.  Over 
a slow international link with 100ms ping times, I was only getting the 
expected 10 TPS doing sync rep with a single client.  But as more 
clients were added, so that a chunk of them were acknowledged in each 
commit reply, the total throughput among all of them scaled near 
linearly.  With 300 clients, that managed to hit a crazy 2000 TPS.

The best scenario to show this patch working would be a laptop drive 
spinning at a slow speed (5400 or 4200 RPM) so that individual local 
commits are slow.  That won't be 100ms slow, but close to 10ms is easy 
to see.  When adding clients to a system with a slow local commit, what 
I've observed is that the scaling levels off between 750 and 1000 TPS, 
no matter how many clients are involved.  The hope is that this 
alternate implementation will give the higher scaling in the face of 
slow commits that is seen on sync rep.

-- 
Greg Smith   2ndQuadrant US    greg@2ndQuadrant.com   Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support  www.2ndQuadrant.us

  4. Re: Group Commit

    Jeff Janes <jeff.janes@gmail.com> — 2011-11-16T00:18:07Z

    On Mon, Nov 14, 2011 at 2:08 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
> Enclosed patch implements Group Commit and also powersave mode for WALWriter.
>
> XLogFlush() waits for WALWriter to run XLogBackgroundFlush(), which
> flushes WAL and then wakes waiters. Uses same concepts and similar
> code to sync rep.
>
> Purpose is to provide consistent WAL writes, even when WALInsertLock
> contended. Currently no "off" option, thinking is that the overhead of
> doing this is relatively low and so it can be "always on" - exactly as
> it is for sync rep.
>
> WALWriter now has variable wakeups, so wal_writer_delay is removed.
> Commit_delay and Commit_siblings are now superfluous and are also removed.
>
> Works, but needs discussion in some areas, docs and possibly tuning
> first, so this is more of a quicky than a slow, comfortable patch.

When I apply this to HEAD and run make check, it freezes at:
test tablespace               ...

The wal writer process is running at 100% CPU.  The only thing it is
doing that is visible though strace is checking that the postmaster is
alive.

I see the same behavior on two different Linux boxes, 32 bit and 64
bit.  I've tried to do some debugging, but haven't made much head-way.
 Does anyone else see this?

Cheers,

Jeff

  5. Re: Group Commit

    Robert Haas <robertmhaas@gmail.com> — 2011-11-16T01:17:43Z

    On Tue, Nov 15, 2011 at 7:18 PM, Jeff Janes <jeff.janes@gmail.com> wrote:
> When I apply this to HEAD and run make check, it freezes at:
> test tablespace               ...
[...]
>  Does anyone else see this?

It hangs for me, too, just slightly later:

============== running regression test queries        ==============
test tablespace               ... ok
parallel group (18 tests):  name txid oid float4 text int2 int4 int8
char varchar uuid float8 money boolean bit enum numeric

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

  6. Re: Group Commit

    Simon Riggs <simon@2ndquadrant.com> — 2011-11-16T08:18:42Z

    On Wed, Nov 16, 2011 at 1:17 AM, Robert Haas <robertmhaas@gmail.com> wrote:
> On Tue, Nov 15, 2011 at 7:18 PM, Jeff Janes <jeff.janes@gmail.com> wrote:
>> When I apply this to HEAD and run make check, it freezes at:
>> test tablespace               ...
> [...]
>>  Does anyone else see this?
>
> It hangs for me, too, just slightly later:


OK, thanks for checking. I'll investigate.


-- 
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services