Re: Online enabling of checksums

Magnus Hagander <magnus@hagander.net>

From: Magnus Hagander <magnus@hagander.net>
To: Andres Freund <andres@anarazel.de>
Cc: Andrey Borodin <x4mmm@yandex-team.ru>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, Heikki Linnakangas <hlinnaka@iki.fi>, Daniel Gustafsson <daniel@yesql.se>, Michael Banck <michael.banck@credativ.de>, Robert Haas <robertmhaas@gmail.com>, Greg Stark <stark@mit.edu>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2018-04-05T21:32:19Z
Lists: pgsql-hackers
On Thu, Apr 5, 2018 at 11:23 PM, Andres Freund <andres@anarazel.de> wrote:

> Hi,
>
> On 2018-04-05 22:06:36 +0200, Magnus Hagander wrote:
> > I have now pushed this latest version with some minor text adjustments
> and
> > a catversion bump.
>
> Is there any sort of locking that guarantees that worker processes see
> an up2date value of
> DataChecksumsNeedWrite()/ControlFile->data_checksum_version? Afaict
> there's not. So you can afaict end up with checksums being computed by
> the worker, but concurrent writes missing them.  The window is going to
> be at most one missed checksum per process (as the unlocking of the page
> is a barrier) and is probably not easy to hit, but that's dangerous
> enough.
>

So just to be clear of the case you're worried about. It's basically:
Session #1 - sets checksums to inprogress
Session #1 - starts dynamic background worker ("launcher")
Launcher reads and enumerates pg_database
Launcher starts worker in first database
Worker processes first block of data in database
And at this point, Session #2 has still not seen the "checksums inprogress"
flag and continues to write without checksums?

That seems like quite a long time to me -- is that really a problem? I'm
guessing you're seeing a shorter path between the two that I can't see
right now (I'll blame the late evning...)?

-- 
 Magnus Hagander
 Me: https://www.hagander.net/ <http://www.hagander.net/>
 Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>

Commits

  1. Online enabling and disabling of data checksums

  2. Deactive flapping checksum isolation tests.

  3. Add support for coordinating record typmods among parallel workers.