Horton

Re: Refactoring the checkpointer's fsync request queue

Shawn Debnath <sdn@amazon.com>

View thread View on postgr.es
From: Shawn Debnath <sdn@amazon.com>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Andres Freund <andres@anarazel.de>, Dmitry Dolgov <9erthalion6@gmail.com>, Robert Haas <robertmhaas@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2019-03-05T16:07:15Z
Lists: pgsql-hackers 
On Tue, Mar 05, 2019 at 10:45:37PM +1300, Thomas Munro wrote:
> On Tue, Mar 5, 2019 at 2:25 PM Shawn Debnath <sdn@amazon.com> wrote:
> > [v11 patch]
> 
> Thanks.  Hmm, something is wrong here because make check is
> dramatically slower -- for example the "insert" test runs in ~8-13
> seconds instead of the usual ~0.2 seconds according to Travis,
> AppVeyor and my local FreeBSD system (note that fsync is disabled so
> it's not that -- it must be bogus queue-related CPU?)

Confirmed. Patch shows 8900 ms vs 192 ms on master for the insert test.  
Interesting! It's reproducible so should be able to figure out what's 
going on. The only thing we do in ForwardSyncRequest() is split up the 8 
bits into 2x4 bits and copy the FileTagData structure to the 
checkpointer queue. Will report back what I find.

-- 
Shawn Debnath
Amazon Web Services (AWS)

Commits

  1. Fix bugs in mdsyncfiletag().

  2. Refactor the fsync queue for wider use.

  3. Don't forget about failed fsync() requests.

  4. PANIC on fsync() failure.

  5. Move LockClauseStrength, LockWaitPolicy into new file nodes/lockoptions.h.

  6. Add new file for checkpointer.c

  7. Split work of bgwriter between 2 processes: bgwriter and checkpointer.

  8. Install infrastructure for shared-memory free space map. Doesn't actually