Re: better page-level checksums

Peter Eisentraut <peter.eisentraut@enterprisedb.com>

From: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
To: Robert Haas <robertmhaas@gmail.com>, Peter Geoghegan <pg@bowt.ie>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2022-06-10T13:36:39Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. Rethink method for assigning OIDs to the template0 and postgres DBs.

  2. pg_upgrade: Preserve database OIDs.

  3. pg_upgrade: Preserve relfilenodes and tablespace OIDs.

  4. Fix for new Boolean node

  5. Improve error handling of HMAC computations

  6. Add macro RelationIsPermanent() to report relation permanence

  7. Enhance nbtree index tuple deletion.

On 10.06.22 15:16, Robert Haas wrote:
> I'm not perfectly attached to the idea of using SHA here, but it seems
> to me that's pretty much the standard thing these days. Stephen Frost
> and David Steele pushed hard for SHA checksums in backup manifests,
> and actually wanted it to be the default.

That seems like a reasonable use in that application, since you might 
want to verify whether a backup has been (maliciously?) altered rather 
than just accidentally bit flipped.

> I think that if you're the kind of person who looks at our existing
> page checksums and finds them too weak, I doubt that CRC-32C is going
> to make you feel any better. You're probably the sort of person who
> thinks that checksums should have a lot of bits, and you're probably
> not going to be satisfied with the properties of an algorithm invented
> in the 1960s. Of course if there's anyone out there who thinks that
> our existing 16-bit checksums are a pile of garbage but would be much
> happier if CRC-32C is an option, I am happy to have them show up here
> and say so, but I find it much more likely that people who want this
> kind of feature would advocate for a more modern algorithm.

I think there ought to be a bit more principled analysis here than just 
"let's add a lot more bits".  There is probably some kind of information 
to be had about how many CRC bits are useful for a given block size, say.

And then there is the question of performance.  When data checksum were 
first added, there was a lot of concern about that.  CRC is usually 
baked directly into hardware, so it's about as cheap as we can hope for. 
  SHA not so much.