Horton

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

Joe Conway <mail@joeconway.com>

View thread
From: Joe Conway <mail@joeconway.com>
To: Masahiko Sawada <sawada.mshk@gmail.com>, Stephen Frost <sfrost@snowman.net>
Cc: Bruce Momjian <bruce@momjian.us>, Antonin Houska <ah@cybertec.at>, Sehrope Sarkuni <sehrope@jackdb.com>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, Robert Haas <robertmhaas@gmail.com>, Haribabu Kommi <kommi.haribabu@gmail.com>, "Moon, Insung" <Moon_Insung_i3@lab.ntt.co.jp>, Ibrar Ahmed <ibrar.ahmad@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2019-08-26T10:49:18Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Revamp the WAL record format.

On 8/26/19 2:53 AM, Masahiko Sawada wrote:
> I guess that this depends on the number of encryption keys we use. If
> we have encryption keys per tablespace or database the number of keys
> would be at most several dozen or several hundred. It's enough to have
> them in flat-file format on the disk and to load them to the hash
> table on the shared memory. We would not need a complex mechanism.
> OTOH if we have keys per tables, we would need to consider indexes and
> buffering as they might not fit in the memory.

Master key(s) need to be kept in memory, but derived keys (using KDF)
would be calculated at time of use, I would think.

>> Some kind of flat-file based approach with a temp file and renaming of
>> files using durable_rename(), like what we used to do with
>> pg_shadow/authid, and now do with replorigin_checkpoint and such?
> 
> The PoC patch I created does that for the keyring file. When key
> rotation, the correspond WAL contains all re-encrypted keys with the
> master key identifier, and the recovery restores the keyring file. One
> good point of this approach is that external tools and startup process
> read it easier. It doesn't require backend codes such as system cache
> and heap functions.

That sounds like a good approach.

Joe

-- 
Crunchy Data - http://crunchydata.com
PostgreSQL Support for Secure Enterprises
Consulting, Training, & Open Source Development