Horton

Re: Key management with tests

Bruce Momjian <bruce@momjian.us>

View thread
From: Bruce Momjian <bruce@momjian.us>
To: Stephen Frost <sfrost@snowman.net>
Cc: Masahiko Sawada <sawada.mshk@gmail.com>, Tom Kincaid <tomjohnkincaid@gmail.com>, Robert Haas <robertmhaas@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Andres Freund <andres@anarazel.de>, Amit Kapila <amit.kapila16@gmail.com>, Thomas Munro <thomas.munro@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>, Masahiko Sawada <masahiko.sawada@2ndquadrant.com>
Date: 2021-03-12T03:31:28Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Rethink method for assigning OIDs to the template0 and postgres DBs.

  2. pg_upgrade: Preserve database OIDs.

  3. pg_upgrade: Preserve relfilenodes and tablespace OIDs.

  4. Fix for new Boolean node

  5. Improve error handling of HMAC computations

  6. Add macro RelationIsPermanent() to report relation permanence

  7. Enhance nbtree index tuple deletion.

Attachments

I have made significant progress on the cluster file encryption feature so
it is time for me to post a new set of patches.

I went backward and forward on this patch.  I went backward and created
significant user and developer documentation for this feature.  I went
forward and added heap/index file encryption, so it actually does
something useful.

Patch
-----
I have broken the patch into 13 parts, attached, which can be reviewed
separately.  However, to get a running system, you have to apply all the
patches in order.  The GitHub patch link has more details, has a combined
patch link, and is updated regularly:

	https://wiki.postgresql.org/wiki/Transparent_Data_Encryption#Patches

I have 13 questions about the patch's approach, marked with XXX
comments.  I hope to get these 13 patches into committable shape, but
not commit them, and then continue on to WAL and temporary file
encryption, and pg_basebackup changes.  My hope is to have this full
feature committed together early in the PG 15 development cycle.  This
progress would not be possible without critical work by Masahiko Sawada
and Stephen Frost.

Performance
-----------
I have used pgbench to test the performance overhead of this feature.
AES128 shows 2.5% overhead, and AES256 shows 3.5% overhead.  My testing
was done on this 16-core, 24MB, single SSD server:

	https://momjian.us/main/blogs/pgblog/2012.html#January_20_2012

My CPU supports AES acceleration.  I made these configuration changes:

	shared_buffers = 6GB
	effective_io_concurrency = 256
	effective_cache_size = 12GB
	checkpoint_completion_target = 0.9
	max_wal_size = 3GB
	wal_compression = on

I used a scale factor of 1000 (15GB database), with this invocation:

	pgbench --no-vacuum --protocol prepared --client 32 --jobs 16 --time 3600

I prewarmed the server via pg_prewarm and ran pgbench for an hour before
using the results from another one-hour run.  TPS was around 6.5k.  For
the default shared_buffers size of 128MB, I see 7% overhead.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee