Horton

Re: Refactoring the checkpointer's fsync request queue

Robert Haas <robertmhaas@gmail.com>

View thread View on postgr.es
From: Robert Haas <robertmhaas@gmail.com>
To: Andres Freund <andres@anarazel.de>
Cc: Thomas Munro <thomas.munro@enterprisedb.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>, sdn@amazon.com, Dmitry Dolgov <9erthalion6@gmail.com>
Date: 2018-11-16T15:05:44Z
Lists: pgsql-hackers 
On Wed, Nov 14, 2018 at 4:49 PM Andres Freund <andres@anarazel.de> wrote:
> On 2018-11-14 16:36:49 -0500, Robert Haas wrote:
> > But how do you make reading that counter atomic with the open() itself?
>
> I don't see why it has to be. As long as the "fd generation" assignment
> happens before fsync (and writes secondarily), there ought not to be any
> further need for synchronizity?

If the goal is to have the FD that is opened first end up in the
checkpointer's table, grabbing a counter backwards does not achieve
it, because there's a race.

S1: open FD
S2: open FD
S2: local_counter = shared_counter++
S1: local_counter = shared_counter++

Now S1 was opened first but has a higher shared counter value than S2
which was opened later.  Does that matter?  Beats me!  I just work
here...

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Commits

  1. Fix bugs in mdsyncfiletag().

  2. Refactor the fsync queue for wider use.

  3. Don't forget about failed fsync() requests.

  4. PANIC on fsync() failure.

  5. Move LockClauseStrength, LockWaitPolicy into new file nodes/lockoptions.h.

  6. Add new file for checkpointer.c

  7. Split work of bgwriter between 2 processes: bgwriter and checkpointer.

  8. Install infrastructure for shared-memory free space map. Doesn't actually