Re: Key management with tests

On Fri, Jan 15, 2021 at 04:56:24PM -0800, Andres Freund wrote:
> On 2021-01-15 19:21:32 -0500, Bruce Momjian wrote:
> > You have to understand cryptography and Postgres internals to understand
> > the design, and I don't think it is realistic to explain that all to the
> > community.  We did much of this in voice calls over months because it
> > was too much of a burden to explain all the cryptographic details so
> > everyone could follow along.
> 
> I think that's not at all acceptable. I don't mind hashing out details
> on calls / off-list, but the design needs to be public, documented, and
> reviewable.  And if it's something the community can't understand, then
> it can't get in. We're going to have to maintain this going forward.

OK, so we don't want it.  That's fine with me.

> I don't mean to say that we need to re-hash all design details from
> scratch - but that there needs to be an explanation somewhere that
> describes what's being done on a medium-high level, and what drove those
> design decisions.

I thought the TODO list was that, and the email threads.

> > > The wiki page doesn't really describe a design either. It has a very
> > > long todo, a bunch of implementation details, but no design.
> > 
> > I am not sure what design document you are requesting.  I thought the
> > TODO was that.
> 
> The TODO in https://wiki.postgresql.org/wiki/Transparent_Data_Encryption#Other_requirements
> is a design document?

Yes.

> > > Nor did 978f869b99 include much in the way of design description.
> > > 
> > > You cannot expect anybody to review a patch if developing some basic
> > > understanding of the intended design requires reading hundreds of
> > > messages in which the design evolved. And I don't think it's acceptable
> > > to push it due to lack of further feedback, given this situation - the
> > > lack of design description is a blocker in itself.
> > 
> > OK, I will just move on to something else then.  It is not worth the
> > feature to go into that kind of discussion again.  I am willing to have
> > voice calls with individuals to explain the logic, but repeatedly
> > explaining it to the entire group I find unproductive.  I don't think
> > another 400-email thread would help anyone.
> 
> Explaining something over voice doesn't help with people in a year or
> five trying to understand the code and the design, so they can adapt it
> when making half-related changes. Nor do I see why another 400 email
> thread would be a necessary consequence of you explaining the design
> that you came up with.

I have underestimated the amount of discussion this has required
repeatedly, and I don't want to make that mistake again.

> This isn't specific to this topic? I don't really understand why this
> specific feature gets to avoid normal community development processes?

What is being avoided?

> > > - tests:
> > >   - wait, a .sh test script? No, we shouldn't add any more of those,
> > >     they're nightmare across platforms
> > 
> > The script originatad from pg_upgrade.  I don't know how to do things
> > like initdb and stuff another way, at least in our code.
> 
> We have had perl tap tests for quite a while now? And all new tests that
> aren't regression / isolation tests are expected to be written in it.

What Perl tap tests run initdb and manage the cluster?  I didn't find
any.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee