Horton

Re: Enable data checksums by default

Daniel Gustafsson <daniel@yesql.se>

View thread
From: Daniel Gustafsson <daniel@yesql.se>
To: Peter Eisentraut <peter@eisentraut.org>
Cc: Greg Sabino Mullane <htamfids@gmail.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Date: 2024-08-08T13:01:57Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Avoid BufferGetLSNAtomic() calls during nbtree scans.

  2. doc PG 18 relnotes: Add incompatibility note about checksums now default

  3. Fix pg_upgrade's cross-version tests when old < 18

  4. initdb: Change default to using data checksums.

  5. Allow TAP tests to force checksums off when calling init()

  6. initdb: Add new option "--no-data-checksums"

  7. Tweak docs to reduce possible impact of data checksums

> On 8 Aug 2024, at 12:11, Peter Eisentraut <peter@eisentraut.org> wrote:

> My understanding was that the reason for some hesitation about adopting data checksums was the performance impact.  Not the checksumming itself, but the overhead from hint bit logging.  The last time I looked into that, you could get performance impacts on the order of 5% tps.  Maybe that's acceptable, and you of course can turn it off if you want the extra performance.  But I think this should be discussed in this thread.

That's been my experience as well, the overhead of the checksumming is
negligible but the overhead in WAL can be (having hint bits WAL logged does
carry other benefits as well to be fair).

> I think we need to think through the upgrade experience a bit more.

+1

> Unfortunately, pg_checksums hasn't gotten to the point that we were perhaps once hoping for that you could enable checksums on a live system.  

I don't recall there being any work done (or plans for) using pg_checksums on a
live system.  Anyone interested in enabling checksums on a live cluster can
however review the patch for that in:

  https://postgr.es/m/E07A611B-9CF3-4FDB-8CE8-A221E39040EC@yesql.se

> I'm thinking pg_upgrade could have a mode where it adds the checksum during the upgrade as it copies the files (essentially a subset of pg_checksums).  I think that would be useful for that middle tier of users who just want a good default experience.

As a side-note, I implemented this in pg_upgrade at Greenplum (IIRC it was
submitted to -hackers at the time as well) and it worked well with not a lot of
code.

--
Daniel Gustafsson