Re: better page-level checksums

Andrey Borodin <x4m@double.cloud>

From: Andrey Borodin <x4m@double.cloud>

To: Matthias van de Meent <boekewurm+postgres@gmail.com>

Cc: Robert Haas <robertmhaas@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>

Date: 2022-06-10T08:21:43Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Rethink method for assigning OIDs to the template0 and postgres DBs.
- 2cb1272445d2 15.0 landed
pg_upgrade: Preserve database OIDs.
- aa01051418f1 15.0 landed
pg_upgrade: Preserve relfilenodes and tablespace OIDs.
- 9a974cbcba00 15.0 landed
Fix for new Boolean node
- cf925936ecc0 15.0 cited
Improve error handling of HMAC computations
- 5513dc6a304d 15.0 cited
Add macro RelationIsPermanent() to report relation permanence
- 95d77149c535 14.0 landed
Enhance nbtree index tuple deletion.
- d168b666823b 14.0 cited

On Fri, Jun 10, 2022 at 5:00 AM Matthias van de Meent <
boekewurm+postgres@gmail.com> wrote:

>
> Can't we add some extra fork that stores this extra per-page
> information, and contains this extra metadata
>

+1 for this approach. I had observed some painful corruption cases where
block storage simply returned stale version of a rage of blocks. This is
only possible because checksum is stored on the page itself.
A special fork for checksums would allow us to better detect failures in
SSD firmawares, MMU SEUs etc, OS page cache, backup software and storage.
It may seems that these kind of stuff never happen. But probability of such
failure is drastically bigger than probability of hardware failure being
undetected due to CRC16 collision.

Also I'm skeptical about correcting detected errors with the information
from checksum. This approach requires very very large checksum. It's much
easier to obtain fresh block copy from HA standby.

Best regards, Andrey Borodin.