Re: Online enabling of checksums

Tomas Vondra <tomas.vondra@2ndquadrant.com>

From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Magnus Hagander <magnus@hagander.net>, Andres Freund <andres@anarazel.de>
Cc: Robert Haas <robertmhaas@gmail.com>, PostgreSQL Developers <pgsql-hackers@lists.postgresql.org>, Peter Eisentraut <peter.eisentraut@2ndquadrant.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>, Daniel Gustafsson <daniel@yesql.se>
Date: 2018-02-25T21:38:58Z
Lists: pgsql-hackers
On 02/25/2018 03:57 PM, Magnus Hagander wrote:
> 
> ...
> 
>> > I very strongly doubg it's a "very noticeable operational problem". People
>> > > don't restart their databases very often... Let's say it takes 2-3 weeks
>> > to
>> > > complete a run in a fairly large database. How many such large databases
>> > > actually restart that frequently? I'm not sure I know of any. And the
>> > only
>> > > effect of it is you have to start the process over (but read-only for the
>> > > part you have already done). It's certainly not ideal, but I don't agree
>> > > it's in any form a "very noticeable problem".
>> >
>> > I definitely know large databases that fail over more frequently than
>> > that.
>> >
>>
>> I would argue that they have bigger issues than enabling checksums... By
>> far.
> 
> In one case it's intentional, to make sure the overall system copes. Not
> that insane.
> 
> That I can understand. But in a scenario like that, you can also stop
> doing that for the period of time when you're rebuilding checksums, if
> re-reading the database over and over again is an issue.
> 
> Note, I'm not saying it wouldn't be nice to have the incremental
> functionality. I'm just saying it's not needed in a first version.
> 

I agree with this sentiment. I don't think we can make each patch
perfect for everyone - certainly not in v1 :-/

Sure, it would be great to allow resume after a restart, but if that
means we won't get anything in PG 11 then I think it's not a good
service to our users. OTOH if the patch without a resume addresses the
issue for 99% of users, and we can improve it in PG 12, why not? That
seems exactly like the incremental thing we do for many other features.

So +1 to not make the "incremental resume" mandatory. If we can support
it, great! But I think the patch may seem less complex than it actually
is, and figuring out how the resume should work will take some time.

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


Commits

  1. Online enabling and disabling of data checksums

  2. Deactive flapping checksum isolation tests.

  3. Add support for coordinating record typmods among parallel workers.