Re: Scaling XLog insertion (was Re: Moving more work outside WALInsertLock)

Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>

From: Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>

To: Fujii Masao <masao.fujii@gmail.com>

Cc: Jeff Janes <jeff.janes@gmail.com>, Robert Haas <robertmhaas@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>

Date: 2012-02-17T15:36:58Z

Lists: pgsql-hackers

Attachments

xloginsert-scale-9.patch (text/x-diff) patch

On 17.02.2012 07:27, Fujii Masao wrote:
> Got another problem: when I ran pg_stop_backup to take an online backup,
> it got stuck until I had generated new WAL record. This happens because,
> in the patch, when pg_stop_backup forces a switch to new WAL file, old
> WAL file is not marked as archivable until next new WAL record has been
> inserted, but pg_stop_backup keeps waiting for that WAL file to be archived.
> OTOH, without the patch, WAL file is marked as archivable as soon as WAL
> file switch occurs.
>
> So, in short, the patch seems to handle the WAL file switch logic incorrectly.

Yep. For a WAL-switch record, XLogInsert returns the location of the end 
of the record, not the end of the empty padding space. So when the 
caller flushed up to that point, it didn't flush the empty space and 
therefore didn't notify the archiver.

Attached is a new version, fixing that, and off-by-one bug you pointed 
out in the slot wraparound handling. I also moved code around a bit, I 
think this new division of labor between the XLogInsert subroutines is 
more readable.

Thanks for the testing!

-- 
   Heikki Linnakangas
   EnterpriseDB   http://www.enterprisedb.com