kms_doc_v10.patch

diff --git a/doc/src/sgml/key-management.sgml b/doc/src/sgml/key-management.sgml
new file mode 100644
index 0000000000..9195316a5e
--- /dev/null
+++ b/doc/src/sgml/key-management.sgml
@@ -0,0 +1,280 @@
+<!-- doc/src/sgml/key-management.sgml -->
+
+<chapter id="key-management">
+ <title>Encryption Key Management</title>
+
+ <indexterm zone="key-management">
+  <primary>key management</primary>
+ </indexterm>
+
+ <para>
+  <productname>PostgreSQL</productname> supports internal
+  <firstterm>Encryption Key Management System</firstterm>, which is designed to manage 
+  the life cycles of cryptographic keys within the PostgreSQL system. This includes dealing 
+  with their generation, storage, usage and rotation. 
+ </para>
+ 
+ <para>
+  Encryption Key Management System is enabled when <productname>PostgreSQL</productname> is
+  built with <literal>--with-openssl</literal> and <xref linkend="app-initdb-cluster-passphrase-command"/> 
+  is specified during <command>initdb</command>. The cluster passphrase provided by
+  <option>--cluster-passphrase-command</option> option during <command>initdb</command>
+  and the one generated by <xref linkend="guc-cluster-passphrase-command"/> in the
+  <filename>postgresql.conf</filename> must match, otherwise, the database cluster
+  will not start up. Please note that the cluster passphrase command passed to <command>initdb</command>
+  must return a passphrase equal or longer than 64 bytes and less than 1024 bytes. For example.
+  <screen>
+   initdb -D dbname --cluster-passphrase-command="echo 1234567812345678123456781234567812345678123456781234567812345678"
+  </screen>
+ </para>
+
+ <sect1 id="key-management-generations-and-derivations">
+  <title>Key Generations and Derivations</title>
+
+  <para>
+   When cluster_passphrase_command option is specified to the initdb, the process will 
+   derive the cluster passphrase into a Key Encryption Key (KEK) and a HMAC Key using key 
+   derivation protocol before the actual generation of application level cryptographic 
+   level keys. 
+  </para>
+  
+  <variablelist>
+   <varlistentry>
+    <term><literal>Key Encryption Key (KEK)</literal></term>
+    <listitem>
+     <para>
+      KEK is primarily used to encapsulate or restore a given application level cryptographic 
+      key
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><literal>HMAC Key</literal></term>
+    <listitem>
+     <para>
+      HMAC key is used to compute the HASH of a given application level cryptographic key for 
+      integrity check purposes. <literal>SHA256</literal> is the algorithm used along with the 
+	  HMAC key to compute a cryptographic hash for integrity check purposes.
+     </para>
+    </listitem>
+   </varlistentry>
+  </variablelist>
+
+  <para>
+   These 2 keys are not stored physically within the PostgreSQL cluster as they are designed 
+   to be derived from the correctly configured cluster passphrase. 
+  </para>
+  
+  <para>
+   Encryption Key Management System currently manages 3 application level cryptographic keys 
+   that have different purposes and usages within the PostgreSQL system and these are generated 
+   using pg_strong_random() after KEK and HMAC key derivation during initdb process. The 3 keys
+   are:
+  </para>
+  
+  <variablelist>
+   <varlistentry>
+    <term><literal>SQL Level Key</literal></term>
+    <listitem>
+     <para>
+      SQL Level Key is used to wrap and unwrap a user secret / passphrase via pg_wrap() and pg_unwrap()
+      SQL functions. These 2 functions are designed to be used in conjunction with the cryptographic
+      functions provided by pgcrypto extension to perform column level encryption/decryption without 
+      having to supply a clear text user secret or passphrase that is required by many pgcrypto 
+      functions as input. Please refer to [Wrap and Unwrap User Secret section] for usage examples.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><literal>Block Level Key</literal></term>
+    <listitem>
+     <para>
+      Block Level Key is primarily used to encrypt / decrypt buffers as part of the Transparent Data 
+      Encryption (TDE) feature
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><literal>WAL Level Key</literal></term>
+    <listitem>
+     <para>
+      WAL Level Key is primarily used to encrypt / decrypt WAL files as part of the Transparent Data 
+      Encryption (TDE) feature
+     </para>
+    </listitem>
+   </varlistentry>
+  </variablelist>
+ </sect1>
+ 
+ <sect1 id="key-management-initialization">
+  <title>Key Initialization</title>
+  
+  <para>
+   When a PostgreSQL cluster with encryption key management enabled is started, the 
+   cluster_passphrase_command parameter in postgresql.conf will be evaluated and the cluster 
+   passphrase will be derived into KEK and HMAC Key in similar ways as initdb. 
+  </para> 
+  
+  <para>
+   After that, the 3 encapsulated application level cryptographic keys will be retrieved from 
+   pg_cryptokeys directory to be restored and integrity-checked by the key management system 
+   using the derived KEK and HMAC key. If this process fails, it is likely that the cluster 
+   passphrase supplied to the cluster is not the same as that supplied to the initdb process.
+   The cluster will refuse to start in this case and user has to manually correct the cluster 
+   passphrase.
+  </para>  
+ </sect1> 
+ 
+ <sect1 id="key-management-wrap-and-unwrap">
+  <title>Wrap and Unwrap User Secret</title>
+
+  <para>
+   Encryption key management system provides pg_wrap() and pg_unwrap SQL functions described in 
+   <xref linkend="functions-key-management-table"/> to perform wrap and unwrap operations on user 
+   secret with the SQL level encryption key. The SQL level encryption key is one of the 3 application 
+   level keys generated during initdb process when cluster_passphrase is supplied.
+  </para> 
+  
+  <para>
+   When <function>pg_wrap()</function> and <function>pg_unwrap()</function> functions are invoked, SQL 
+   level encryption key will internally be used to perform the encryption and decryption operation with 
+   HMAC-based integrity check. From user's point of view, he or she is not aware of the actual SQL level 
+   encryption key used internally by both wrap functions.
+  </para>  
+  
+  <para>
+   One possible use case is to combine <function>pg_wrap()</function> and <function>pg_unwrap()</function>
+   with <xref linkend="pgcrypto"/>. User wraps the user encryption secret with <function>pg_wrap</function> 
+   function and passes the wrapped encryption secret to <function>pg_unwrap</function> function for the 
+   pgcrypto encryption functions. The wrapped secret can be stored in the application server or somewhere 
+   secured and should be obtained promptly for cryptographic operation with <structname>pgcrypto</structname>.
+  </para>  
+  
+  <para>
+   Here is an example that shows how to encrypt and decrypt data together with
+   wrap and unwrap functions:
+  </para>
+
+<programlisting>
+=# SELECT pg_wrap('my secret passward');
+                                                                              pg_wrap
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ \xb2c89f76f04f95d029f179e0fc3df4ed7254127b5562a9e27d42d1cd037c942dea65ce7c0750c520fa4f4e90481c9eb7e1e42a068248c262c1a6f25c6eab64303b1154ccc9a14361223641aab4a7aabe
+(1 row)
+</programlisting>
+
+  <para>
+   Once wrapping the user key, user can encrypt and decrypt user data using the
+   wrapped user key together with the key unwrap functions:
+  </para>
+
+<programlisting>
+ =# INSERT INTO tbl
+        VALUES (pgp_sym_encrypt('secret data',
+                                 pg_unwrap('\xb2c89f76f04f95d029f179e0fc3df4ed7254127b5562a9e27d42d1cd037c942dea65ce7c0750c520fa4f4e90481c9eb7e1e42a068248c262c1a6f25c6eab64303b1154ccc9a14361223641aab4a7aabe')));
+ INSERT 1
+
+ =# SELECT * FROM tbl;
+                                                                             col
+--------------------------------------------------------------------------------------------------------------------------------------------------------------
+ \xc30d04070302a199ee38bea0320b75d23c01577bb3ffb315d67eecbeca3e40e869cea65efbf0b470f805549af905f94d94c447fbfb8113f585fc86b30c0bd784b10c9857322dc00d556aa8de14
+(1 row)
+
+ =# SELECT pgp_sym_decrypt(col,
+                           pg_unwrap('\xb2c89f76f04f95d029f179e0fc3df4ed7254127b5562a9e27d42d1cd037c942dea65ce7c0750c520fa4f4e90481c9eb7e1e42a068248c262c1a6f25c6eab64303b1154ccc9a14361223641aab4a7aabe')) as col
+    FROM tbl;
+     col
+--------------
+ secret data
+(1 row)
+</programlisting>
+
+  <para>
+   The data <literal>'secret data'</literal> is practically encrypted by the
+   user secret <literal>'my secret passward'</literal> but using wrap and
+   unwrap functions user don't need to know the actual user secret during
+   operation.
+  </para>
+ </sect1>
+
+ <sect1 id="key-management-rotation">
+  <title>Key Rotation Process</title>
+
+  <para>
+   Encryption keys in general are not interminable, the longer the same key is in use, 
+   the chance  of it being breached increases. Performing key rotation on a regular basis 
+   help meet standardized security practices such as <ulink url="https://www.pcisecuritystandards.org/">
+   PCI-DSS</ulink> and it is a good practice in security to limit the number of encrypted 
+   bytes available for a specific key version. The key lifetimse are based on key length, 
+   key strength, algorithm and total number of bytes enciphered. The key management systems 
+   provides a efficient method to perform key rotation. 
+  </para>
+  
+  <para>
+   Please be aware that the phrase <literal>"key rotation"</literal> here only refers to the 
+   rotation of KEK and HMAC keys. The 3 application level encryption keys (SQL, Block and WAL 
+   levels) are not rotated; they will in fact be the same before and after a <literal>"key 
+   rotation."</literal> This can be justified because the actual keys are never stored anywhere 
+   physically, presented to user or captured in logging. What is being rotated here is the KEK 
+   and HMAC keys who are responsible for encapsulating and restoring the actual application level 
+   encryption keys. 
+  </para>
+  
+  <para>
+   Since both KEK and HMAC keys are derived from a cluster passphrase, the <literal>"key rotation"
+   </literal> ultimately refers to the rotation of cluster passphrase and deriving a new KEK and 
+   HMAC keys from the new cluster passphrase. The new set of KEK and HMAC keys can then be used to 
+   encapsulate all 3 application level encryptions keys and store the new results in pg_cryptokeys 
+   directory.
+  </para>
+  
+  <para>
+   To complete the cluster passphrase rotation, user needs to follow the steps below:
+  </para>
+  <itemizedlist>
+   <listitem>
+    <para>
+     Ensure the PostgreSQL server is running correctly with KMS enabled. Passphrase rotation cannot
+	 be completed with the server shut down.
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+     Update <literal>cluster_passphrase_command</literal> parameter in the <filename>postgresql.conf</filename>
+	 such that the new command will return a new cluster passphrase.
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+	In a <literal>psql</literal> session, execute <function>pg_rotate_cluster_passphrase()</function> SQL 
+     function to initiate the rotation. The function returns true upon successful key rotation and false if
+	 otherwise.
+    </para>
+	<programlisting>
+postgres=> SELECT pg_rotate_cluster_passphrase();
+pg_rotate_cluster_passphrase 
+------------------------------
+ t
+(1 row)
+    </programlisting>
+   </listitem>
+  </itemizedlist>
+  <para>
+   Upon successful cluster passphrase rotation, all application level keys (SQL, WAL and Block Levels) will be 
+   re-encapsulated by the new KEK and hashed by the new HMAC key derived from the new cluster passphrase. The
+   new encapsulated application level keys will be stored in pg_cryptokeys directory. Please note that the 
+   actual application keys are the same as before; the rotation process only changes the KEK and HMAC key that 
+   are used to encapsulate and verify the actual application level keys. This way, there is no need to decrypt 
+   all the encrypted data with the old keys and re-encrypt them with the new. 
+  </para>
+  <para>
+   In case of a crash during the cluster passphrase rotation process, the key management system is able to recover
+   to the previous sets of application level keys the next time server starts up. This is possible because the
+   key rotation and encapsulation process are done on a separate tempory key directory called <literal>
+   pg_cryptokeys_tmp</literal> and it will replace <literal>pg_cryptokeys</literal> and be deleted only when 
+   everything is successfully finished. If the server starts with pg_cryptokeys_tmp folder present, it would
+   indicate that previous attempt of cluster passphrase rotation was not completed. In this case, the server
+   will discard pg_cryptokeys_tmp folder and load the keys in pg_cryptokeys as usual.
+  </para>
+ </sect1>
+</chapter>