Re: Offline enabling/disabling of data checksums
Fabien COELHO <coelho@cri.ensmp.fr>
From: Fabien COELHO <coelho@cri.ensmp.fr>
To: Magnus Hagander <magnus@hagander.net>
Cc: Tomas Vondra <tomas.vondra@2ndquadrant.com>,
Michael Paquier <michael@paquier.xyz>,
Michael Banck <michael.banck@credativ.de>,
Postgres hackers <pgsql-hackers@postgresql.org>
Date: 2018-12-28T14:50:55Z
Lists: pgsql-hackers
>>> [...] >> >> I'm not sure data checksums are particularly great evidence. For example >> with the recent fsync issues, we might have ended with partial writes >> (and thus invalid checksums). The OS migh have even told us about the >> failure, but we've gracefully ignored it. So I'm afraid data checksums >> are not a particularly great proof it's not our fault. > > They are a great evidence that your data is corrupt. You *want* to know > that your data is corrupt. Even if our best recommendation is "go restore > your backups", you still want to know. Otherwise you are sitting around on > data that's corrupt and you don't know about it. > > There are certainly many things we can do to improve the experience. But > not telling people their data is coorrupt when it is, isn't one of them. Yep, anyone should want to know if their database is corrupt, compare to ignoring the fact. One reason not to enable it could be if the implementation is not trusted, i.e. if false positive (corrupt page detected while the data are okay and there was only an issue with computing or storing the checksum) can occur. There is also the performance impact. I did some quick-and-dirty pgbench simple update single thread performance tests to compare with vs without checksum. Enabling checksums on these tests seems to induce a 1.4% performance penalty, although I'm moderately confident about it given the standard deviation. At least it is an indication, and it seems to me that it is consistent with other figures previously reported on the list. -- Fabien.
Commits
-
Reorganize Notes section in documentation of pg_checksums
- a7cc52370b2b 12.0 landed
-
Add option -N/--no-sync to pg_checksums
- e0090c869008 12.0 landed
-
Add options to enable and disable checksums in pg_checksums
- ed308d783790 12.0 landed
-
Refactor more code logic to update the control file
- 8b938d36f744 12.0 landed
-
Rename pg_verify_checksums to pg_checksums
- 6dd263cfaa84 12.0 landed
-
Fix cross-version compatibility checks of pg_verify_checksums
- 501f58359b59 11.3 landed
- c9ae7f704c67 12.0 landed
-
Add routine able to update the control file to src/common/
- ce6afc6823fb 12.0 landed