Horton

Re: better page-level checksums

Peter Eisentraut <peter.eisentraut@enterprisedb.com>

View thread
From: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
To: Robert Haas <robertmhaas@gmail.com>, Aleksander Alekseev <aleksander@timescale.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Stephen Frost <sfrost@snowman.net>, Peter Geoghegan <pg@bowt.ie>, Andrey Borodin <x4mmm@yandex-team.ru>
Date: 2022-06-15T08:54:50Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Rethink method for assigning OIDs to the template0 and postgres DBs.

  2. pg_upgrade: Preserve database OIDs.

  3. pg_upgrade: Preserve relfilenodes and tablespace OIDs.

  4. Fix for new Boolean node

  5. Improve error handling of HMAC computations

  6. Add macro RelationIsPermanent() to report relation permanence

  7. Enhance nbtree index tuple deletion.

On 13.06.22 20:20, Robert Haas wrote:
> If the user wants 16-bit checksums, the feature we've already got
> seems good enough -- and, as you say, it doesn't use any extra disk
> space. This proposal is just about making people happy if they want a
> bigger checksum.

It's hard to get any definite information about what size of checksum is 
"good enough", since after all it depends on what kinds of errors you 
expect and what kinds of probabilities you want to accept.  But the best 
I could gather so far is that 16-bit CRC are good until about 16 kB 
block size.

Which leads to the question whether there is really a lot of interest in 
catering to larger block sizes.  The recent thread about performance 
impact of different block sizes might renew interest in this.  But 
unless we really want to encourage playing with the block sizes (and if 
my claim above is correct), then a larger checksum size might not be needed.

> On the topic of which algorithm to use, I'd be inclined to think that
> it is going to be more useful to offer checksums that are 64 bits or
> more, since IMHO 32 is not all that much more than 16, and I still
> think there are going to be alignment issues. Beyond that I don't have
> anything against your specific suggestions, but I'd like to hear what
> other people think.

Again, gathering some vague information ...

The benefits of doubling the checksum size are exponential rather than 
linear, so there is no significant benefit of using a 64-bit checksum 
over a 32-bit one, for supported block sizes (current max is 32 kB).