Horton

Re: storing an explicit nonce

Bruce Momjian <bruce@momjian.us>

View thread
From: Bruce Momjian <bruce@momjian.us>
To: Andres Freund <andres@anarazel.de>
Cc: Robert Haas <robertmhaas@gmail.com>, Stephen Frost <sfrost@snowman.net>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Masahiko Sawada <sawada.mshk@gmail.com>, Tom Kincaid <tomjohnkincaid@gmail.com>, Amit Kapila <amit.kapila16@gmail.com>, Thomas Munro <thomas.munro@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>, Masahiko Sawada <masahiko.sawada@2ndquadrant.com>
Date: 2021-05-25T19:07:38Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Rethink method for assigning OIDs to the template0 and postgres DBs.

  2. pg_upgrade: Preserve database OIDs.

  3. pg_upgrade: Preserve relfilenodes and tablespace OIDs.

  4. Fix for new Boolean node

  5. Improve error handling of HMAC computations

  6. Add macro RelationIsPermanent() to report relation permanence

  7. Enhance nbtree index tuple deletion.

On Tue, May 25, 2021 at 10:37:32AM -0700, Andres Freund wrote:
> The obvious concerns are issues around binary upgrades for cases that
> already use the special space? Are you planning to address that by not
> having that path? Or by storing the nonce at the "start" of the special
> space (i.e. [normal data][nonce][existing special])?
> 
> Is there an argument for generalizing the nonce approach for to replace
> fake LSNs for unlogged relations?
> 
> Why is using pd_special better than finding space for a flag bit in the
> header indicating whether it has a nonce? Using pd_special will burden
> all code using special space, and maybe even some that does not (think
> empty pages now occasionally having a non-zero pd_special), whereas
> implementing it on the page level wouldn't quite have the same concerns.

My code can already identify if the LSN is fake or not --- why can't we
build on that?  Can someone show that logging WAL hint bits causes
unacceptable overhead beyond the encryption overhead?  I don't think we
even know that since we don't know the overhead of encrypting WAL.

One crazy idea would be to not log WAL hints, but rather use an LSN
range that will never be valid for real LSNs, like the high bit being
set.  That special range would need to be WAL-logged, but again, perhaps
every 1k, and increment by 1k on a crash.

This discussion has cemented what I had already considered --- that
doing a separate nonce will make this feature less usable/upgradable,
and take it beyond my ability or desire to complete.

Ideally, what I would like to do is to resolve my XXX questions in my
patches, get everyone happy with what we have, then let me do the WAL
encryption.  We can then see if logging hint bits is significant
overhead, and if it is, go with a special LSN range for fake LSNs.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  If only the physical world exists, free will is an illusion.