Horton

Re: better page-level checksums

Robert Haas <robertmhaas@gmail.com>

View thread
From: Robert Haas <robertmhaas@gmail.com>
To: Peter Geoghegan <pg@bowt.ie>
Cc: Bruce Momjian <bruce@momjian.us>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2022-06-14T15:47:49Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Rethink method for assigning OIDs to the template0 and postgres DBs.

  2. pg_upgrade: Preserve database OIDs.

  3. pg_upgrade: Preserve relfilenodes and tablespace OIDs.

  4. Fix for new Boolean node

  5. Improve error handling of HMAC computations

  6. Add macro RelationIsPermanent() to report relation permanence

  7. Enhance nbtree index tuple deletion.

On Mon, Jun 13, 2022 at 6:26 PM Peter Geoghegan <pg@bowt.ie> wrote:
> Anyway, I can see how it would be useful to be able to know the offset
> of a nonce or of a hash digest on any given page, without access to a
> running server. But why shouldn't that be possible with other designs,
> including designs closer to what I've outlined?

I don't know what you mean by this. As far as I'm aware, the only
design you've outlined is one where the space wasn't at the same
offset on every page.

> A known fixed offset in the special area already assumes that all
> pages must have a value in the first place, even though that won't be
> true for the majority of individual Postgres servers. There is
> implicit information involved in a design like the one Robert has
> proposed; your backup tool (or whatever) already has to understand to
> expect something other than no encryption at all, or no checksum at
> all. Tools like pg_filedump already rely on implicit information about
> the special area.

In general, I was imagining that you'd need to look at the control
file to understand how much space had been reserved per page in this
particular cluster. I agree that's a bit awkward, especially for
pg_filedump. However, pg_filedump and I think also some code internal
to PostgreSQL try to figure out what kind of page we've got by looking
at the *size* of the special space. It's only good luck that we
haven't had a collision there yet, and continuing to rely on that
seems like a dead end. Perhaps we should start including a per-AM
magic number at the beginning of the special space.

> I'm not against the idea of picking a handful of checksum/encryption
> schemes, with the understanding that we'll be committing to those
> particular schemes indefinitely -- it's not reasonable to expect
> infinite flexibility here (and so I don't). But why should we accept
> something that seems to me to be totally inflexible, and doesn't
> compose with other things?

We shouldn't accept something that's totally inflexible, but I don't
know why this seems that way to you.

-- 
Robert Haas
EDB: http://www.enterprisedb.com