Horton

Re: Enable data checksums by default

Daniel Gustafsson <daniel@yesql.se>

View thread
From: Daniel Gustafsson <daniel@yesql.se>
To: Tomas Vondra <tomas@vondra.me>
Cc: Heikki Linnakangas <hlinnaka@iki.fi>, Peter Eisentraut <peter@eisentraut.org>, Nathan Bossart <nathandbossart@gmail.com>, Greg Sabino Mullane <htamfids@gmail.com>, Bruce Momjian <bruce@momjian.us>, Michael Paquier <michael@paquier.xyz>, Michael Banck <mbanck@gmx.net>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Date: 2025-05-23T10:05:37Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Avoid BufferGetLSNAtomic() calls during nbtree scans.

  2. doc PG 18 relnotes: Add incompatibility note about checksums now default

  3. Fix pg_upgrade's cross-version tests when old < 18

  4. initdb: Change default to using data checksums.

  5. Allow TAP tests to force checksums off when calling init()

  6. initdb: Add new option "--no-data-checksums"

  7. Tweak docs to reduce possible impact of data checksums

> On 23 May 2025, at 11:55, Tomas Vondra <tomas@vondra.me> wrote:
> 
> On 5/23/25 11:25, Daniel Gustafsson wrote:
>>> On 23 May 2025, at 10:10, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
>> 
>>> Aside from just documenting it, I see two things we could do:
>>> 
>>> 1. Have pg_upgrade run initdb for you. It's always felt silly that you need to run initdb with the new version yourself, when there's really only one correct way to do it. pg_upgrade has all the checks to verify that you did it right, so why doesn't it just do it itself? I think that'd be a good long-term solution. Might be too late for 18, but I'm not sure. If someone wrote the patch we could evaluate it. To use that mode, the scripts calling pg_upgrade would need to be changed, though, so we'd perhaps want to do #2 or something else in addition to this.
>> 
>> I can see this being desired longer term, but as you mention there is likely to
>> be many moving parts outside of our immediate control making it much harder
>> than just adding the call to initdb.  It doesn't seem like a post-beta patch to
>> me given the implications for packagers and others in the ecosystem.
>> 
>>> 2. If the new cluster has checksums enabled, but the old one has them disabled, have pg_upgrade disable checksums in the new cluster.
>> 
>> IF we do this it should be Very visible, since a user otherwise might think
>> that their upgraded cluster will have checksums since they added them in
>> initdb.
> 
> What counts as "very visible"? Would it be fine if the pg_upgrade docs
> say this clearly, and pg_upgrade prints a warning? To me that seems
> sufficient.

I was thinking about a warning during processing.

> TBH I can't quite imagine people expecting checksums to just magically
> appear after upgrade.

It would not be surprised if users expect checksums to be on after reading
(variations of) "checksums are now on by default" messaging.

>> I think we should document how to deal with checksums in upgrades, and perhaps
>> even tweak the errormessage in the pg_upgrade check with explanatory comments
>> if needed, and leave the functionality as is today.
> 
> Isn't that just an unnecessary breakage of existing tooling? I mean,
> there's pretty much just one thing the user can do to make it work, and
> that's disabling checksums. Sure, they might also enable checksums on
> the old cluster, but that makes the upgrade much longer, and presumably
> they use pg_upgrade to upgrade quickly.

We already expect the new cluster to be created in the Right Way (which I agree
isn't very userfriendly and should be improved upon) so requiring this to be
Right is in line with existing tooling IMHO (for better or worse).  My concern
is that users will think data checksums are enabled after the upgrade, and will
be annoyed when finding out they're not.

--
Daniel Gustafsson