Horton

Re: storing an explicit nonce

Robert Haas <robertmhaas@gmail.com>

View thread
From: Robert Haas <robertmhaas@gmail.com>
To: Bruce Momjian <bruce@momjian.us>
Cc: Stephen Frost <sfrost@snowman.net>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Masahiko Sawada <sawada.mshk@gmail.com>, Tom Kincaid <tomjohnkincaid@gmail.com>, Andres Freund <andres@anarazel.de>, Amit Kapila <amit.kapila16@gmail.com>, Thomas Munro <thomas.munro@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>, Masahiko Sawada <masahiko.sawada@2ndquadrant.com>
Date: 2021-05-25T19:09:03Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Rethink method for assigning OIDs to the template0 and postgres DBs.

  2. pg_upgrade: Preserve database OIDs.

  3. pg_upgrade: Preserve relfilenodes and tablespace OIDs.

  4. Fix for new Boolean node

  5. Improve error handling of HMAC computations

  6. Add macro RelationIsPermanent() to report relation permanence

  7. Enhance nbtree index tuple deletion.

On Tue, May 25, 2021 at 2:45 PM Bruce Momjian <bruce@momjian.us> wrote:
> Well, if we create a separate nonce counter, we still need to make sure
> it doesn't go backwards during a crash, so we have to WAL log it

I think we don't really need a global counter, do we? We could simply
increment the nonce every time we write the page. If we want to avoid
using the same IV for different pages, then 8 bytes of the nonce could
store a value that's different for every page, and the other 8 bytes
could store a counter. Presumably we won't manage to write the same
page more than 2^64 times, since LSNs are limited to be <2^64, and
those are consumed more than 1 byte at a time for every change to any
page anywhere.

> The buffer encryption overhead is 2-4%, and WAL encryption is going to
> add to that, so I thought hint bit logging overhead would be minimal
> in comparison.

I think it depends. If buffer evictions are rare, then it won't matter
much. But if they are common, then using the LSN as the nonce will add
a lot of overhead.

> Have you looked at the code, specifically EncryptPage():
>
>         https://github.com/postgres/postgres/compare/bmomjian:cfe-11-gist..bmomjian:_cfe-12-rel.patch
>
> +       if (!relation_is_permanent && !is_gist_page_or_similar)
> +               PageSetLSN(page, LSNForEncryption(relation_is_permanent));
>
>
> It assigns an LSN to unlogged pages.  As far as the buffer manager
> seeing fake LSNs that already happens for GiST indexes, so I just built
> on that --- seemed to work fine.

I had not, but I don't see why this issue is specific to GiST rather
than common to every kind of unlogged and temporary relation.

> I have to ask why we should consider adding it to the special space,
> since my current version seems fine, and has minimal code impact, and
> has some advantages over using the special space.  Is it because of the
> WAL hint overhead, or for a cleaner API, or something else?

My concern is about the overhead, and also the code complexity. I
think that making sure that the LSN gets changed in all cases may be
fairly tricky.

-- 
Robert Haas
EDB: http://www.enterprisedb.com