Horton

Re: Refactoring the checkpointer's fsync request queue

Robert Haas <robertmhaas@gmail.com>

View thread View on postgr.es
From: Robert Haas <robertmhaas@gmail.com>
To: Thomas Munro <thomas.munro@enterprisedb.com>
Cc: Andres Freund <andres@anarazel.de>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>, sdn@amazon.com, Dmitry Dolgov <9erthalion6@gmail.com>
Date: 2018-11-14T21:36:49Z
Lists: pgsql-hackers 
On Tue, Nov 13, 2018 at 6:44 PM Thomas Munro
<thomas.munro@enterprisedb.com> wrote:
> > That sounds a little like you are proposing to go back to the way
> > things were before 806a2aee3791244bf0f916729bfdb5489936e068 (and,
> > belatedly, bf405ba8e460051e715d0a91442b579e590328ce) although I guess
> > the division of labor wouldn't be quite the same.
>
> But is there an argument against it?  The checkpointer would still be
> creating checkpoints including running fsync, but the background
> writer would be, erm, writing, erm, in the background.

I don't know.  I guess the fact that the checkpointer is still
performing the fsyncs is probably a key point.  I mean, in the old
division of labor, fsyncs could interrupt the background writing that
was supposed to be happening.

> I'm not sure if it matters whether we send the fd before or after the
> write, but we still need some kind of global ordering of fds that can
> order a given fd with respect to writes in other processes, so the
> patch introduces a global shared counter captured immediately after
> open() (including when reopened in the vfd machinery).

But how do you make reading that counter atomic with the open() itself?

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Commits

  1. Fix bugs in mdsyncfiletag().

  2. Refactor the fsync queue for wider use.

  3. Don't forget about failed fsync() requests.

  4. PANIC on fsync() failure.

  5. Move LockClauseStrength, LockWaitPolicy into new file nodes/lockoptions.h.

  6. Add new file for checkpointer.c

  7. Split work of bgwriter between 2 processes: bgwriter and checkpointer.

  8. Install infrastructure for shared-memory free space map. Doesn't actually