Horton

Thread

Commits

  1. Add regression expected-files for older OpenSSL in FIPS mode.

  2. Allow tests to pass in OpenSSL FIPS mode (rest)

  3. Allow tests to pass in OpenSSL FIPS mode (TAP tests)

  4. pgcrypto: Allow tests to pass in OpenSSL FIPS mode

  5. pgcrypto: Split off pgp-encrypt-md5 test

  6. citext: Allow tests to pass in OpenSSL FIPS mode

  7. Remove incidental md5() function uses from main regression tests

  8. Improve/correct comments

  9. Put tests of md5() function into separate test file

  1. Allow tests to pass in OpenSSL FIPS mode

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2022-10-04T15:45:32Z

    While working on the column encryption patch, I wanted to check that 
what is implemented also works in OpenSSL FIPS mode.  I tried running 
the normal test suites after switching the OpenSSL installation to FIPS 
mode, but that failed all over the place.  So I embarked on fixing that. 
  Attached is a first iteration of a patch.

The main issue is liberal use of the md5() function in tests to generate 
random strings.  For example, this is a common pattern:

     SELECT x, md5(x::text) FROM generate_series(-10,10) x;

This can be replaced by

     SELECT x, encode(sha256(x::text::bytea), 'hex')
         FROM generate_series(-10,10) x;

In most cases, this could be further simplified by not using text but 
bytea for the column types, thus skipping the encode step.

Some tests are carefully calibrated to achieve a certain column size or 
something like that.  These will need to be checked in more detail.

Another set of issues is in the SSL tests, where apparently some 
certificates are generated with obsolete hash methods, probably SHA1 
(and possibly MD5 again).  Some of this can be addressed by just 
regenerating everything with a newer OpenSSL installation, in some other 
cases it appears to need additional command-line options or a local 
configuration file change.  This needs more research.  I think we should 
augment the setup used to generate these test files in a way that they 
don't depend on the local configuration of whoever runs it.

Of course, there are some some tests where we do want to test MD5 
functionality, such as in the authentication tests or in the tests of 
the md5() function itself.  I think we can conditionalize these somehow. 
  That looks like a smaller issue compared to the issues above.

  2. Re: Allow tests to pass in OpenSSL FIPS mode

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2022-10-11T11:51:50Z

    On 04.10.22 17:45, Peter Eisentraut wrote:
> While working on the column encryption patch, I wanted to check that 
> what is implemented also works in OpenSSL FIPS mode.  I tried running 
> the normal test suites after switching the OpenSSL installation to FIPS 
> mode, but that failed all over the place.  So I embarked on fixing that. 

> Of course, there are some some tests where we do want to test MD5 
> functionality, such as in the authentication tests or in the tests of 
> the md5() function itself.  I think we can conditionalize these somehow. 

Let's make a small start on this.  The attached patch moves the tests of 
the md5() function to a separate test file.  That would ultimately make 
it easier to maintain a variant expected file for FIPS mode where that 
function will fail (similar to how we have done it for the pgcrypto tests).

  3. Re: Allow tests to pass in OpenSSL FIPS mode

    Michael Paquier <michael@paquier.xyz> — 2022-10-12T01:18:50Z

    On Tue, Oct 11, 2022 at 01:51:50PM +0200, Peter Eisentraut wrote:
> Let's make a small start on this.  The attached patch moves the tests of the
> md5() function to a separate test file.  That would ultimately make it
> easier to maintain a variant expected file for FIPS mode where that function
> will fail (similar to how we have done it for the pgcrypto tests).

Makes sense to me.  This slice looks fine.

I think that the other md5() computations done in the main regression
test suite could just be switched to use one of the sha*() functions
as they just want to put their hands on text values.  It looks like a
few of them have some expections with the output size and
generate_series(), though, but this could be tweaked by making the
series shorter, for example.
--
Michael

  4. Re: Allow tests to pass in OpenSSL FIPS mode

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2022-10-13T10:26:32Z

    On 12.10.22 03:18, Michael Paquier wrote:
> On Tue, Oct 11, 2022 at 01:51:50PM +0200, Peter Eisentraut wrote:
>> Let's make a small start on this.  The attached patch moves the tests of the
>> md5() function to a separate test file.  That would ultimately make it
>> easier to maintain a variant expected file for FIPS mode where that function
>> will fail (similar to how we have done it for the pgcrypto tests).
> 
> Makes sense to me.  This slice looks fine.

Committed.

> I think that the other md5() computations done in the main regression
> test suite could just be switched to use one of the sha*() functions
> as they just want to put their hands on text values.  It looks like a
> few of them have some expections with the output size and
> generate_series(), though, but this could be tweaked by making the
> series shorter, for example.

Right, that's the rest of my original patch.  I'll come back with an 
updated version of that.

  5. Re: Allow tests to pass in OpenSSL FIPS mode

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2022-10-13T11:16:18Z

    On 2022-Oct-13, Peter Eisentraut wrote:

> Right, that's the rest of my original patch.  I'll come back with an updated
> version of that.

However, there are some changes in brin_multi.out that are quite
surprising and suggest that we might have bugs in brin:

+WARNING:  unexpected number of results 31 for (macaddr8col,>,macaddr8,b1:d1:0e:7b:af:a4:42:12,33)
+WARNING:  unexpected number of results 17 for (macaddr8col,>=,macaddr8,d9:35:91:bd:f7:86:0e:1e,15)
+WARNING:  unexpected number of results 11 for (macaddr8col,<=,macaddr8,23:e8:46:63:86:07:ad:cb,13)
+WARNING:  unexpected number of results 4 for (macaddr8col,<,macaddr8,13:16:8e:6a:2e:6c:84:b4,6)

-- 
Álvaro Herrera        Breisgau, Deutschland  —  https://www.EnterpriseDB.com/
"La victoria es para quien se atreve a estar solo"

  6. Re: Allow tests to pass in OpenSSL FIPS mode

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2022-12-07T14:14:09Z

    On 13.10.22 12:26, Peter Eisentraut wrote:
>> I think that the other md5() computations done in the main regression
>> test suite could just be switched to use one of the sha*() functions
>> as they just want to put their hands on text values.  It looks like a
>> few of them have some expections with the output size and
>> generate_series(), though, but this could be tweaked by making the
>> series shorter, for example.
> 
> Right, that's the rest of my original patch.  I'll come back with an 
> updated version of that.

Here is the next step.  To contain the scope, I focused on just "make 
check" for now.  This patch removes all incidental calls to md5(), 
replacing them with sha256(), so that they'd pass with or without FIPS 
mode.  (Two tests would need alternative expected files: md5 and 
password.  I have not included those here.)

Some tests inspect the actual md5 result strings or build statistics 
based on them.  I have tried to carefully preserve the meaning of the 
original tests, to the extent that they could be inferred, in some cases 
adjusting example values by matching the md5 outputs to the equivalent 
sha256 outputs.  Some cases are tricky or mysterious or both and could 
use another look.

  7. Re: Allow tests to pass in OpenSSL FIPS mode

    Michael Paquier <michael@paquier.xyz> — 2022-12-09T04:16:08Z

    On Wed, Dec 07, 2022 at 03:14:09PM +0100, Peter Eisentraut wrote:
> Here is the next step.  To contain the scope, I focused on just "make check"
> for now.  This patch removes all incidental calls to md5(), replacing them
> with sha256(), so that they'd pass with or without FIPS mode.  (Two tests
> would need alternative expected files: md5 and password.  I have not
> included those here.)

Yeah, fine by me to do that step-by-step.

> Some tests inspect the actual md5 result strings or build statistics based
> on them.  I have tried to carefully preserve the meaning of the original
> tests, to the extent that they could be inferred, in some cases adjusting
> example values by matching the md5 outputs to the equivalent sha256 outputs.
> Some cases are tricky or mysterious or both and could use another look.

incremental_sort mostly relies on the plan generated, so the change
should be rather straight-forward I guess, though there may be a side
effect depending on costing.  Hmm, it does not look like stats_ext
would be an issue as it checks the stats correlation of the attributes
for mcv_lists_arrays.

largeobject_1.out has been forgotten in the set requiring a refresh.
--
Michael

  8. Re: Allow tests to pass in OpenSSL FIPS mode

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2023-01-31T09:55:30Z

    On 09.12.22 05:16, Michael Paquier wrote:
>> Some tests inspect the actual md5 result strings or build statistics based
>> on them.  I have tried to carefully preserve the meaning of the original
>> tests, to the extent that they could be inferred, in some cases adjusting
>> example values by matching the md5 outputs to the equivalent sha256 outputs.
>> Some cases are tricky or mysterious or both and could use another look.
> incremental_sort mostly relies on the plan generated, so the change
> should be rather straight-forward I guess, though there may be a side
> effect depending on costing.  Hmm, it does not look like stats_ext
> would be an issue as it checks the stats correlation of the attributes
> for mcv_lists_arrays.
> 
> largeobject_1.out has been forgotten in the set requiring a refresh.

Here is a refreshed patch with the missing file added.

  9. Re: Allow tests to pass in OpenSSL FIPS mode

    Michael Paquier <michael@paquier.xyz> — 2023-02-27T07:16:00Z

    On Thu, Oct 13, 2022 at 01:16:18PM +0200, Alvaro Herrera wrote:
> However, there are some changes in brin_multi.out that are quite
> surprising and suggest that we might have bugs in brin:
> 
> +WARNING:  unexpected number of results 31 for (macaddr8col,>,macaddr8,b1:d1:0e:7b:af:a4:42:12,33)
> +WARNING:  unexpected number of results 17 for (macaddr8col,>=,macaddr8,d9:35:91:bd:f7:86:0e:1e,15)
> +WARNING:  unexpected number of results 11 for (macaddr8col,<=,macaddr8,23:e8:46:63:86:07:ad:cb,13)
> +WARNING:  unexpected number of results 4 for (macaddr8col,<,macaddr8,13:16:8e:6a:2e:6c:84:b4,6)

This refers to brin_minmax_multi_distance_macaddr8(), no?  This is
amazing.  I have a hard time imagining how FIPS would interact with
what we do in mac8.c to explain that, so it may be something entirely
different.  Is that reproducible?
--
Michael

  10. Re: Allow tests to pass in OpenSSL FIPS mode

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2023-02-27T07:23:34Z

    On 27.02.23 08:16, Michael Paquier wrote:
> On Thu, Oct 13, 2022 at 01:16:18PM +0200, Alvaro Herrera wrote:
>> However, there are some changes in brin_multi.out that are quite
>> surprising and suggest that we might have bugs in brin:
>>
>> +WARNING:  unexpected number of results 31 for (macaddr8col,>,macaddr8,b1:d1:0e:7b:af:a4:42:12,33)
>> +WARNING:  unexpected number of results 17 for (macaddr8col,>=,macaddr8,d9:35:91:bd:f7:86:0e:1e,15)
>> +WARNING:  unexpected number of results 11 for (macaddr8col,<=,macaddr8,23:e8:46:63:86:07:ad:cb,13)
>> +WARNING:  unexpected number of results 4 for (macaddr8col,<,macaddr8,13:16:8e:6a:2e:6c:84:b4,6)
> 
> This refers to brin_minmax_multi_distance_macaddr8(), no?  This is
> amazing.  I have a hard time imagining how FIPS would interact with
> what we do in mac8.c to explain that, so it may be something entirely
> different.  Is that reproducible?

This is no longer present in the v2 patch.

  11. Re: Allow tests to pass in OpenSSL FIPS mode

    Michael Paquier <michael@paquier.xyz> — 2023-02-28T05:01:04Z

    On Mon, Feb 27, 2023 at 08:23:34AM +0100, Peter Eisentraut wrote:
> On 27.02.23 08:16, Michael Paquier wrote:
>> This refers to brin_minmax_multi_distance_macaddr8(), no?  This is
>> amazing.  I have a hard time imagining how FIPS would interact with
>> what we do in mac8.c to explain that, so it may be something entirely
>> different.  Is that reproducible?
> 
> This is no longer present in the v2 patch.

Sure, but why was it happening in the first place?  The proposed patch
set only reworks some regression tests.  So It seems to me that this
is a sign that we may have issues in some code area that got stressed
in some new way, no?
--
Michael

  12. Re: Allow tests to pass in OpenSSL FIPS mode

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2023-02-28T07:25:00Z

    On 28.02.23 06:01, Michael Paquier wrote:
> On Mon, Feb 27, 2023 at 08:23:34AM +0100, Peter Eisentraut wrote:
>> On 27.02.23 08:16, Michael Paquier wrote:
>>> This refers to brin_minmax_multi_distance_macaddr8(), no?  This is
>>> amazing.  I have a hard time imagining how FIPS would interact with
>>> what we do in mac8.c to explain that, so it may be something entirely
>>> different.  Is that reproducible?
>>
>> This is no longer present in the v2 patch.
> 
> Sure, but why was it happening in the first place?

Because the earlier patch only changed the test input values (which were 
generated on the fly using md5()), but did not adjust the expected test 
results in all the places.

  13. Re: Allow tests to pass in OpenSSL FIPS mode

    Tom Lane <tgl@sss.pgh.pa.us> — 2023-03-04T23:04:37Z

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
> [ v2-0001-Remove-incidental-md5-function-uses-from-main-reg.patch ]

I've gone through this and have a modest suggestion: let's invent some
wrapper functions around encode(sha256()) to reduce the cosmetic diffs
and consequent need for closer study of patch changes.  In the attached
I called them "notmd5()", but I'm surely not wedded to that name.

This also accounts for some relatively recent additions to stats_ext.sql
that introduced yet more uses of md5().  This passes for me on a
FIPS-enabled Fedora system, with the exception of md5.sql and
password.sql.  I agree that the right thing for md5.sql is just to add
a variant expected-file.  password.sql could perhaps use some refactoring
so that we don't have two large expected-files to manage.

The only other place that perhaps needs discussion is rowsecurity.sql,
which has some surprisingly large changes: not only do the random
strings change, but there are rowcount differences in some results.
I believe this is because there are RLS policy checks and view conditions
that actually examine the contents of the "md5" strings, eg

CREATE POLICY p1 ON s1 USING (a in (select x from s2 where y like '%2f%'));

My recommendation is to just accept those changes as OK and move on.
I doubt that anybody checked the existing results line-by-line either.

So, once we've done something about md5.sql and password.sql, I think
this is committable.

			regards, tom lane

  14. Re: Allow tests to pass in OpenSSL FIPS mode

    Daniel Gustafsson <daniel@yesql.se> — 2023-03-06T09:02:55Z

    > On 5 Mar 2023, at 00:04, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> 
> Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
>> [ v2-0001-Remove-incidental-md5-function-uses-from-main-reg.patch ]
> 
> I've gone through this and have a modest suggestion: let's invent some
> wrapper functions around encode(sha256()) to reduce the cosmetic diffs
> and consequent need for closer study of patch changes.  In the attached
> I called them "notmd5()", but I'm surely not wedded to that name.

For readers without all context, wouldn't it be better to encode in the
function name why we're not just calling a hash like md5?  Something like
fips_allowed_hash() or similar?

--
Daniel Gustafsson

  15. Re: Allow tests to pass in OpenSSL FIPS mode

    Tom Lane <tgl@sss.pgh.pa.us> — 2023-03-06T14:55:06Z

    Daniel Gustafsson <daniel@yesql.se> writes:
>> On 5 Mar 2023, at 00:04, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> I've gone through this and have a modest suggestion: let's invent some
>> wrapper functions around encode(sha256()) to reduce the cosmetic diffs
>> and consequent need for closer study of patch changes.  In the attached
>> I called them "notmd5()", but I'm surely not wedded to that name.

> For readers without all context, wouldn't it be better to encode in the
> function name why we're not just calling a hash like md5?  Something like
> fips_allowed_hash() or similar?

I'd prefer shorter than that --- all these queries are laid out on the
expectation of a very short function name.  Maybe "fipshash()"?

We could make the comment introducing the function declarations more
elaborate, too.

			regards, tom lane

  16. Re: Allow tests to pass in OpenSSL FIPS mode

    Daniel Gustafsson <daniel@yesql.se> — 2023-03-06T16:06:22Z

    > On 6 Mar 2023, at 15:55, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Daniel Gustafsson <daniel@yesql.se> writes:

>> For readers without all context, wouldn't it be better to encode in the
>> function name why we're not just calling a hash like md5?  Something like
>> fips_allowed_hash() or similar?
> 
> I'd prefer shorter than that --- all these queries are laid out on the
> expectation of a very short function name.  Maybe "fipshash()"?
> 
> We could make the comment introducing the function declarations more
> elaborate, too.

fipshash() with an explanatory comments sounds like a good idea.

--
Daniel Gustafsson

  17. Re: Allow tests to pass in OpenSSL FIPS mode

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2023-03-08T07:34:35Z

    On 05.03.23 00:04, Tom Lane wrote:
> I've gone through this and have a modest suggestion: let's invent some
> wrapper functions around encode(sha256()) to reduce the cosmetic diffs
> and consequent need for closer study of patch changes.  In the attached
> I called them "notmd5()", but I'm surely not wedded to that name.

Do you mean create this on the fly in the test suite, or make it a new 
built-in function?

  18. Re: Allow tests to pass in OpenSSL FIPS mode

    Tom Lane <tgl@sss.pgh.pa.us> — 2023-03-08T07:40:37Z

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
> On 05.03.23 00:04, Tom Lane wrote:
>> I've gone through this and have a modest suggestion: let's invent some
>> wrapper functions around encode(sha256()) to reduce the cosmetic diffs
>> and consequent need for closer study of patch changes.  In the attached
>> I called them "notmd5()", but I'm surely not wedded to that name.

> Do you mean create this on the fly in the test suite, or make it a new 
> built-in function?

The former --- please read my version of the patch.

			regards, tom lane

  19. Re: Allow tests to pass in OpenSSL FIPS mode

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2023-03-08T08:49:15Z

    On 09.12.22 05:16, Michael Paquier wrote:
> On Wed, Dec 07, 2022 at 03:14:09PM +0100, Peter Eisentraut wrote:
>> Here is the next step.  To contain the scope, I focused on just "make check"
>> for now.  This patch removes all incidental calls to md5(), replacing them
>> with sha256(), so that they'd pass with or without FIPS mode.  (Two tests
>> would need alternative expected files: md5 and password.  I have not
>> included those here.)
> 
> Yeah, fine by me to do that step-by-step.

It occurred to me that it would be easier to maintain this in the long 
run if we could enable a "fake FIPS" mode that would have the same 
effect but didn't require fiddling with the OpenSSL configuration or 
installation.

The attached patch shows how this could work.  Thoughts?

  20. Re: Allow tests to pass in OpenSSL FIPS mode

    Daniel Gustafsson <daniel@yesql.se> — 2023-03-08T09:21:26Z

    > On 8 Mar 2023, at 09:49, Peter Eisentraut <peter.eisentraut@enterprisedb.com> wrote:

> It occurred to me that it would be easier to maintain this in the long run if we could enable a "fake FIPS" mode that would have the same effect but didn't require fiddling with the OpenSSL configuration or installation.
> 
> The attached patch shows how this could work.  Thoughts?

- * Initialize a hash context.  Note that this implementation is designed
- * to never fail, so this always returns 0.
+ * Initialize a hash context.
Regardless of which, we wan't this hunk since the code clearly can return -1.

+#ifdef FAKE_FIPS_MODE
I'm not enthusiastic about this.  If we use this rather than OpenSSL with FIPS
enabled we might end up missing bugs or weird behavior due to changes in
OpenSSL that we didn't test.

--
Daniel Gustafsson

  21. Re: Allow tests to pass in OpenSSL FIPS mode

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2023-03-08T09:26:54Z

    On 08.03.23 08:40, Tom Lane wrote:
> Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
>> On 05.03.23 00:04, Tom Lane wrote:
>>> I've gone through this and have a modest suggestion: let's invent some
>>> wrapper functions around encode(sha256()) to reduce the cosmetic diffs
>>> and consequent need for closer study of patch changes.  In the attached
>>> I called them "notmd5()", but I'm surely not wedded to that name.
> 
>> Do you mean create this on the fly in the test suite, or make it a new
>> built-in function?
> 
> The former --- please read my version of the patch.

Ok, that makes sense.  We have some other uses of this pattern in other 
test suites that my initial patch didn't cover yet, for example in 
src/test/subscripton, but we don't have expected files there, so the 
argument of reducing the diffs doesn't apply.

  22. Re: Allow tests to pass in OpenSSL FIPS mode

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2023-03-08T09:28:00Z

    On 06.03.23 17:06, Daniel Gustafsson wrote:
> fipshash() with an explanatory comments sounds like a good idea.

I think that name would be quite false advertising.

  23. Re: Allow tests to pass in OpenSSL FIPS mode

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2023-03-08T09:30:21Z

    On 08.03.23 10:21, Daniel Gustafsson wrote:
>> On 8 Mar 2023, at 09:49, Peter Eisentraut <peter.eisentraut@enterprisedb.com> wrote:
> 
>> It occurred to me that it would be easier to maintain this in the long run if we could enable a "fake FIPS" mode that would have the same effect but didn't require fiddling with the OpenSSL configuration or installation.
>>
>> The attached patch shows how this could work.  Thoughts?
> 
> - * Initialize a hash context.  Note that this implementation is designed
> - * to never fail, so this always returns 0.
> + * Initialize a hash context.
> Regardless of which, we wan't this hunk since the code clearly can return -1.

I was a bit puzzled by these comments in that file.  While the existing 
implementations (mostly) never fail, they are clearly not *designed* to 
never fail, since the parallel OpenSSL implementations can fail (which 
is the point of this thread).  So I would remove these comments 
altogether, really.

> +#ifdef FAKE_FIPS_MODE
> I'm not enthusiastic about this.  If we use this rather than OpenSSL with FIPS
> enabled we might end up missing bugs or weird behavior due to changes in
> OpenSSL that we didn't test.

Valid point.  In any case, the patch is available for ad hoc testing.

  24. Re: Allow tests to pass in OpenSSL FIPS mode

    Daniel Gustafsson <daniel@yesql.se> — 2023-03-08T09:37:12Z

    > On 8 Mar 2023, at 10:30, Peter Eisentraut <peter.eisentraut@enterprisedb.com> wrote:
> 
> On 08.03.23 10:21, Daniel Gustafsson wrote:
>>> On 8 Mar 2023, at 09:49, Peter Eisentraut <peter.eisentraut@enterprisedb.com> wrote:
>>> It occurred to me that it would be easier to maintain this in the long run if we could enable a "fake FIPS" mode that would have the same effect but didn't require fiddling with the OpenSSL configuration or installation.
>>> 
>>> The attached patch shows how this could work.  Thoughts?
>> - * Initialize a hash context.  Note that this implementation is designed
>> - * to never fail, so this always returns 0.
>> + * Initialize a hash context.
>> Regardless of which, we wan't this hunk since the code clearly can return -1.
> 
> I was a bit puzzled by these comments in that file.  While the existing implementations (mostly) never fail, they are clearly not *designed* to never fail, since the parallel OpenSSL implementations can fail (which is the point of this thread).  So I would remove these comments altogether, really.

The comment in question was missed in 55fe26a4b58, but I agree that it's a
false claim given the OpenSSL implementation so removing or at least mimicking
the comments in cryptohash_openssl.c would be better.

--
Daniel Gustafsson

  25. Re: Allow tests to pass in OpenSSL FIPS mode

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2023-03-09T09:01:14Z

    On 08.03.23 10:37, Daniel Gustafsson wrote:
> The comment in question was missed in 55fe26a4b58, but I agree that it's a
> false claim given the OpenSSL implementation so removing or at least mimicking
> the comments in cryptohash_openssl.c would be better.

I have fixed these comments to match cryptohash_openssl.c.

  26. Re: Allow tests to pass in OpenSSL FIPS mode

    Michael Paquier <michael@paquier.xyz> — 2023-03-09T10:29:47Z

    On Thu, Mar 09, 2023 at 10:01:14AM +0100, Peter Eisentraut wrote:
> I have fixed these comments to match cryptohash_openssl.c.

Missed that, thanks for the fix.
--
Michael

  27. Re: Allow tests to pass in OpenSSL FIPS mode

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2023-03-13T10:06:42Z

    On 06.03.23 17:06, Daniel Gustafsson wrote:
>> On 6 Mar 2023, at 15:55, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Daniel Gustafsson <daniel@yesql.se> writes:
> 
>>> For readers without all context, wouldn't it be better to encode in the
>>> function name why we're not just calling a hash like md5?  Something like
>>> fips_allowed_hash() or similar?
>>
>> I'd prefer shorter than that --- all these queries are laid out on the
>> expectation of a very short function name.  Maybe "fipshash()"?
>>
>> We could make the comment introducing the function declarations more
>> elaborate, too.
> 
> fipshash() with an explanatory comments sounds like a good idea.

committed like that

(I'm going to close the CF item and revisit the other test suites for 
the next release.)

  28. Re: Allow tests to pass in OpenSSL FIPS mode

    Daniel Gustafsson <daniel@yesql.se> — 2023-03-13T10:10:13Z

    > On 13 Mar 2023, at 11:06, Peter Eisentraut <peter.eisentraut@enterprisedb.com> wrote:
> On 06.03.23 17:06, Daniel Gustafsson wrote:

>> fipshash() with an explanatory comments sounds like a good idea.
> 
> committed like that

+1. Looks like there is a just a slight diff in the compression.sql test suite.

--
Daniel Gustafsson

  29. Re: Allow tests to pass in OpenSSL FIPS mode

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2023-10-05T13:44:19Z

    On 04.10.22 17:45, Peter Eisentraut wrote:
> While working on the column encryption patch, I wanted to check that 
> what is implemented also works in OpenSSL FIPS mode.  I tried running 
> the normal test suites after switching the OpenSSL installation to FIPS 
> mode, but that failed all over the place.  So I embarked on fixing that. 
>   Attached is a first iteration of a patch.

Continuing this, we have fixed many issues since.  Here is a patch set 
to fix all remaining issues.

v4-0001-citext-Allow-tests-to-pass-in-OpenSSL-FIPS-mode.patch
v4-0002-pgcrypto-Allow-tests-to-pass-in-OpenSSL-FIPS-mode.patch

These two are pretty straightforward.

v4-0003-Allow-tests-to-pass-in-OpenSSL-FIPS-mode-TAP-test.patch

This one does some delicate surgery and could use some thorough review.

v4-0004-Allow-tests-to-pass-in-OpenSSL-FIPS-mode-rest.patch

This just adds alternative expected files.  The question is mainly just 
whether there are better ways to organize this.

v4-0005-WIP-Use-fipshash-in-brin_multi-test.patch

Here, some previously fixed md5() uses have snuck back in.  I will need 
to track down the origin of this and ask for a proper fix there.  This 
is just included here for completeness.

  30. Re: Allow tests to pass in OpenSSL FIPS mode

    Daniel Gustafsson <daniel@yesql.se> — 2023-10-05T14:17:38Z

    > On 5 Oct 2023, at 15:44, Peter Eisentraut <peter.eisentraut@enterprisedb.com> wrote:
> 
> On 04.10.22 17:45, Peter Eisentraut wrote:
>> While working on the column encryption patch, I wanted to check that what is implemented also works in OpenSSL FIPS mode.  I tried running the normal test suites after switching the OpenSSL installation to FIPS mode, but that failed all over the place.  So I embarked on fixing that.   Attached is a first iteration of a patch.
> 
> Continuing this, we have fixed many issues since.  Here is a patch set to fix all remaining issues.
> 
> v4-0001-citext-Allow-tests-to-pass-in-OpenSSL-FIPS-mode.patch
> v4-0002-pgcrypto-Allow-tests-to-pass-in-OpenSSL-FIPS-mode.patch

+ERROR:  crypt(3) returned NULL

Not within scope here, but I wish we had a better error message here. That's for another patch though clearly.

> v4-0003-Allow-tests-to-pass-in-OpenSSL-FIPS-mode-TAP-test.patch
> 
> This one does some delicate surgery and could use some thorough review.

I don't have a FIPS enabled build handy to test in, but reading the patch I
don't see anything that sticks out apart from very minor comments:

+my $md5_works = ($node->psql('postgres', "select md5('')") == 0);

I think this warrants an explanatory comment for readers not familiar with
FIPS, without that it may seem quite an odd test.

+), 0, 'created user with scram password');

Tiny nitpick, I think we use SCRAM when writing it in text.

> v4-0004-Allow-tests-to-pass-in-OpenSSL-FIPS-mode-rest.patch
> 
> This just adds alternative expected files.  The question is mainly just whether there are better ways to organize this.

Without inventing a new structure for alternative outputs I don't see how.

--
Daniel Gustafsson

  31. Re: Allow tests to pass in OpenSSL FIPS mode

    Tom Lane <tgl@sss.pgh.pa.us> — 2023-10-05T20:04:04Z

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
> Continuing this, we have fixed many issues since.  Here is a patch set 
> to fix all remaining issues.

On the way to testing this, I discovered that we have a usability
regression with recent OpenSSL releases.  The Fedora 35 installation
I used to use for testing FIPS-mode behavior would produce errors like

 select md5('') = 'd41d8cd98f00b204e9800998ecf8427e' AS "TRUE";
- TRUE 
-------
- t
-(1 row)
-
+ERROR:  could not compute MD5 hash: disabled for FIPS

In the shiny new Fedora 38 installation I just set up for the
same purpose, I'm seeing

 select md5('') = 'd41d8cd98f00b204e9800998ecf8427e' AS "TRUE";
- TRUE 
-------
- t
-(1 row)
-
+ERROR:  could not compute MD5 hash: unsupported


This is less user-friendly; moreover it indicates that we're
going to get different output depending on the vintage of
OpenSSL we're testing against, which is going to be a pain for
expected-file maintenance.

I think we need to make an effort to restore the old output
if possible, although I grant that this may be mostly a whim
of OpenSSL's that we can't do much about.

The F35 installation has openssl 1.1.1q, where F38 has
openssl 3.0.9.

			regards, tom lane

  32. Re: Allow tests to pass in OpenSSL FIPS mode

    Tom Lane <tgl@sss.pgh.pa.us> — 2023-10-05T20:55:39Z

    I found another bit of fun we'll need to deal with: on my F38
platform, pgcrypto/3des fails as attached.  Some googling finds
this relevant info:

https://github.com/pyca/cryptography/issues/6875

That is, FIPS deprecation of 3DES is happening even as we speak.
So apparently we'll have little choice but to deal with two
different behaviors for that.

As before, I'm not too pleased with the user-friendliness
of the error:

+ERROR:  encrypt error: Cipher cannot be initialized

That's even less useful to a user than "unsupported".

FWIW, everything else seems to pass with this patchset.
I ran check-world as well as the various "must run manually"
test suites.

			regards, tom lane

  33. Re: Allow tests to pass in OpenSSL FIPS mode

    Peter Eisentraut <peter@eisentraut.org> — 2023-10-06T13:44:40Z

    On 05.10.23 22:04, Tom Lane wrote:
> On the way to testing this, I discovered that we have a usability
> regression with recent OpenSSL releases.  The Fedora 35 installation
> I used to use for testing FIPS-mode behavior would produce errors like

> +ERROR:  could not compute MD5 hash: disabled for FIPS

> In the shiny new Fedora 38 installation I just set up for the
> same purpose, I'm seeing

> +ERROR:  could not compute MD5 hash: unsupported

This makes sense, because the older OpenSSL works basically like

     if (FIPS_mode()) {
         specific_error();
     }

while the new one has all crypto methods in modules, and if you load the 
fips module, then some crypto methods just don't exist.

  34. Re: Allow tests to pass in OpenSSL FIPS mode

    Peter Eisentraut <peter@eisentraut.org> — 2023-10-06T13:46:24Z

    On 05.10.23 22:55, Tom Lane wrote:
> I found another bit of fun we'll need to deal with: on my F38
> platform, pgcrypto/3des fails as attached.  Some googling finds
> this relevant info:
> 
> https://github.com/pyca/cryptography/issues/6875
> 
> That is, FIPS deprecation of 3DES is happening even as we speak.
> So apparently we'll have little choice but to deal with two
> different behaviors for that.

Hmm, interesting, so maybe there should be a new openssl 3.x release at 
the end of the year that addresses this?

  35. Re: Allow tests to pass in OpenSSL FIPS mode

    Peter Eisentraut <peter@eisentraut.org> — 2023-11-14T10:52:42Z

    On 05.10.23 22:55, Tom Lane wrote:
> I found another bit of fun we'll need to deal with: on my F38
> platform, pgcrypto/3des fails as attached.  Some googling finds
> this relevant info:
> 
> https://github.com/pyca/cryptography/issues/6875
> 
> That is, FIPS deprecation of 3DES is happening even as we speak.
> So apparently we'll have little choice but to deal with two
> different behaviors for that.
> 
> As before, I'm not too pleased with the user-friendliness
> of the error:
> 
> +ERROR:  encrypt error: Cipher cannot be initialized
> 
> That's even less useful to a user than "unsupported".
> 
> FWIW, everything else seems to pass with this patchset.
> I ran check-world as well as the various "must run manually"
> test suites.

I've been trying to get some VM set up with the right Red Hat 
environment to be able to reproduce the issues you reported.  But 
somehow switching the OS into FIPS mode messes up the boot environment 
of the VM or something.  So I haven't been able to make progress on this.

I suggest that if there are no other concerns, we proceed with the patch 
set as is for now.

The 3DES deprecation can be addressed by adding another expected file, 
which can easily be supplied by someone having this environment running.

The error message difference in the older OpenSSL version would probably 
need a small bit of coding.  But we can leave that as a separate add-on 
project.

  36. Re: Allow tests to pass in OpenSSL FIPS mode

    Tom Lane <tgl@sss.pgh.pa.us> — 2023-11-14T23:07:38Z

    Peter Eisentraut <peter@eisentraut.org> writes:
> On 05.10.23 22:55, Tom Lane wrote:
>> I found another bit of fun we'll need to deal with: on my F38
>> platform, pgcrypto/3des fails as attached.  Some googling finds
>> this relevant info:
>> https://github.com/pyca/cryptography/issues/6875
>> That is, FIPS deprecation of 3DES is happening even as we speak.
>> So apparently we'll have little choice but to deal with two
>> different behaviors for that.

> I've been trying to get some VM set up with the right Red Hat 
> environment to be able to reproduce the issues you reported.  But 
> somehow switching the OS into FIPS mode messes up the boot environment 
> of the VM or something.  So I haven't been able to make progress on this.

Hm.  I was just using a native install on a microSD card for my
raspberry pi ...

> I suggest that if there are no other concerns, we proceed with the patch 
> set as is for now.

After thinking about it for awhile, I guess I'm okay with only
bothering to provide expected-files for FIPS failures under OpenSSL
3.x (which is how your patch is set up, I believe).  While there are
certainly still LTS platforms with 1.x, we don't have to consider FIPS
mode on them to be a supported case.

I'm more concerned about the 3DES situation.  Fedora might be a bit
ahead of the curve here, but according to the link above, everybody is
supposed to be in compliance by the end of 2023.  So I'd be inclined
to guess that the 3DES-is-rejected case is going to be mainstream
before v17 ships.

> The error message difference in the older OpenSSL version would probably 
> need a small bit of coding.  But we can leave that as a separate add-on 
> project.

It's the *newer* version's message that I'm unhappy about ;-).
But I agree that that's not a reason to hold up applying what's
here.  (In reality, people running FIPS mode are probably pretty
accustomed to seeing this error, so maybe it's not worth the
trouble to improve it.)

			regards, tom lane

  37. Re: Allow tests to pass in OpenSSL FIPS mode

    Daniel Gustafsson <daniel@yesql.se> — 2023-11-15T10:06:12Z

    > On 15 Nov 2023, at 00:07, Tom Lane <tgl@sss.pgh.pa.us> wrote:

> (In reality, people running FIPS mode are probably pretty
> accustomed to seeing this error, so maybe it's not worth the
> trouble to improve it.)

In my experience this holds a lot of truth, this is a common error pattern and
while all improvements to error messages are good, it's not a reason to hold
off this patch.

--
Daniel Gustafsson

  38. Re: Allow tests to pass in OpenSSL FIPS mode

    Peter Eisentraut <peter@eisentraut.org> — 2023-11-15T11:44:36Z

    On 15.11.23 00:07, Tom Lane wrote:
> I'm more concerned about the 3DES situation.  Fedora might be a bit
> ahead of the curve here, but according to the link above, everybody is
> supposed to be in compliance by the end of 2023.  So I'd be inclined
> to guess that the 3DES-is-rejected case is going to be mainstream
> before v17 ships.

Right.  It is curious that I have not found any activity in the OpenSSL 
issue trackers about this.  But if you send me your results file, then I 
can include it in the patch as an alternative expected.

  39. Re: Allow tests to pass in OpenSSL FIPS mode

    Daniel Gustafsson <daniel@yesql.se> — 2023-11-15T14:25:22Z

    > On 15 Nov 2023, at 12:44, Peter Eisentraut <peter@eisentraut.org> wrote:
> 
> On 15.11.23 00:07, Tom Lane wrote:
>> I'm more concerned about the 3DES situation.  Fedora might be a bit
>> ahead of the curve here, but according to the link above, everybody is
>> supposed to be in compliance by the end of 2023.  So I'd be inclined
>> to guess that the 3DES-is-rejected case is going to be mainstream
>> before v17 ships.
> 
> Right.  It is curious that I have not found any activity in the OpenSSL issue trackers about this.  But if you send me your results file, then I can include it in the patch as an alternative expected.

As NIST SP800-131A allows decryption with 3DES and DES I dont think OpenSSL
will do much other than move it to the legacy module where it can be used
opt-in like DES.  SKIPJACK is already disallowed since before but is still
tested with decryption during FIPS validation.

Using an alternative resultsfile to handle platforms which explicitly removes
disallowed ciphers seem like the right choice.

Since the 3DES/DES deprecations aren't limited to FIPS, do we want to do
anything for pgcrypto where we have DES/3DES encryption?  Maybe a doc patch
which mentions the deprecation with a link to the SP could be in order?

--
Daniel Gustafsson

  40. Re: Allow tests to pass in OpenSSL FIPS mode

    Tom Lane <tgl@sss.pgh.pa.us> — 2023-11-15T20:29:16Z

    Daniel Gustafsson <daniel@yesql.se> writes:
> Since the 3DES/DES deprecations aren't limited to FIPS, do we want to do
> anything for pgcrypto where we have DES/3DES encryption?  Maybe a doc patch
> which mentions the deprecation with a link to the SP could be in order?

A docs patch that marks both MD5 and 3DES as deprecated is probably
appropriate, but it seems like a matter for a separate thread and patch.

In the meantime, I've done a pass of review of Peter's v4 patches.
v4-0001 is already committed, so that's not considered here.

v4-0002: I think it is worth splitting up contrib/pgcrypto's
pgp-encrypt test, which has only one test case whose output changes,
and a bunch of others that don't.  v5-0002, attached, does it
like that.  It's otherwise the same as v4.

(It might be worth doing something similar for uuid_ossp's test,
but I have not bothered here.  That test script is stable enough
that I'm not too worried about future maintenance.)

The attached 0003, 0004, 0005 patches are identical to Peter's.
I think that it is possibly worth modifying the password test so that
we don't fail to create the roles, so as to reduce the delta between
password.out and password_1.out (and thereby ease future maintenance
of those files).  However you might disagree, so I split my proposal
out as a separate patch v5-0007-password-test-delta.patch; you can
drop that from the set if you don't like it.

v5-0006-allow-for-disabled-3DES.patch adds the necessary expected
file to make that pass on my Fedora 38 system.

With or without 0007, as you choose, I think it's committable.

			regards, tom lane

  41. Re: Allow tests to pass in OpenSSL FIPS mode

    Peter Eisentraut <peter@eisentraut.org> — 2023-11-17T18:45:56Z

    On 15.11.23 21:29, Tom Lane wrote:
> Daniel Gustafsson <daniel@yesql.se> writes:
>> Since the 3DES/DES deprecations aren't limited to FIPS, do we want to do
>> anything for pgcrypto where we have DES/3DES encryption?  Maybe a doc patch
>> which mentions the deprecation with a link to the SP could be in order?
> 
> A docs patch that marks both MD5 and 3DES as deprecated is probably
> appropriate, but it seems like a matter for a separate thread and patch.
> 
> In the meantime, I've done a pass of review of Peter's v4 patches.
> v4-0001 is already committed, so that's not considered here.
> 
> v4-0002: I think it is worth splitting up contrib/pgcrypto's
> pgp-encrypt test, which has only one test case whose output changes,
> and a bunch of others that don't.  v5-0002, attached, does it
> like that.  It's otherwise the same as v4.
> 
> (It might be worth doing something similar for uuid_ossp's test,
> but I have not bothered here.  That test script is stable enough
> that I'm not too worried about future maintenance.)
> 
> The attached 0003, 0004, 0005 patches are identical to Peter's.
> I think that it is possibly worth modifying the password test so that
> we don't fail to create the roles, so as to reduce the delta between
> password.out and password_1.out (and thereby ease future maintenance
> of those files).  However you might disagree, so I split my proposal
> out as a separate patch v5-0007-password-test-delta.patch; you can
> drop that from the set if you don't like it.
> 
> v5-0006-allow-for-disabled-3DES.patch adds the necessary expected
> file to make that pass on my Fedora 38 system.
> 
> With or without 0007, as you choose, I think it's committable.

All done, thanks.

  42. Re: Allow tests to pass in OpenSSL FIPS mode

    Thomas Munro <thomas.munro@gmail.com> — 2024-04-19T03:50:40Z

    On Sat, Nov 18, 2023 at 7:46 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> All done, thanks.

Probably not this thread's fault, but following the breadcrumbs to the
last thread to touch the relevant test lines in
authentication/001_password, is it expected that we have these
warnings?

psql:<stdin>:1: WARNING:  roles created by regression test cases
should have names starting with "regress_"

  43. Re: Allow tests to pass in OpenSSL FIPS mode

    Tom Lane <tgl@sss.pgh.pa.us> — 2024-04-19T04:00:21Z

    Thomas Munro <thomas.munro@gmail.com> writes:
> Probably not this thread's fault, but following the breadcrumbs to the
> last thread to touch the relevant test lines in
> authentication/001_password, is it expected that we have these
> warnings?

> psql:<stdin>:1: WARNING:  roles created by regression test cases
> should have names starting with "regress_"

I think the policy is that we enforce that for cases reachable
via "make installcheck" (to avoid possibly clobbering global
objects in a live installation), but not for cases only reachable
via "make check", such as TAP tests.  So I'm not that concerned
about this, although if someone is feeling anal enough to rename
the test role I won't stand in the way.

			regards, tom lane

  44. Re: Allow tests to pass in OpenSSL FIPS mode

    Thomas Munro <thomas.munro@gmail.com> — 2024-04-19T04:12:40Z

    On Fri, Apr 19, 2024 at 4:00 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Thomas Munro <thomas.munro@gmail.com> writes:
> > Probably not this thread's fault, but following the breadcrumbs to the
> > last thread to touch the relevant test lines in
> > authentication/001_password, is it expected that we have these
> > warnings?
>
> > psql:<stdin>:1: WARNING:  roles created by regression test cases
> > should have names starting with "regress_"
>
> I think the policy is that we enforce that for cases reachable
> via "make installcheck" (to avoid possibly clobbering global
> objects in a live installation), but not for cases only reachable
> via "make check", such as TAP tests.  So I'm not that concerned
> about this, although if someone is feeling anal enough to rename
> the test role I won't stand in the way.

Got it, thanks.  Not me, just asking.

  45. Re: Allow tests to pass in OpenSSL FIPS mode

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-09-14T19:02:15Z

    [ blast-from-the-past department ]

I wrote:
> Peter Eisentraut <peter@eisentraut.org> writes:
>> I suggest that if there are no other concerns, we proceed with the patch 
>> set as is for now.

> After thinking about it for awhile, I guess I'm okay with only
> bothering to provide expected-files for FIPS failures under OpenSSL
> 3.x (which is how your patch is set up, I believe).  While there are
> certainly still LTS platforms with 1.x, we don't have to consider FIPS
> mode on them to be a supported case.

I see that Mark W. has just spun up a couple of BF animals running
FIPS mode under SLES 15 (goshawk and shoebill).  Not too surprisingly,
they are failing the MD5 test:

 select md5('') = 'd41d8cd98f00b204e9800998ecf8427e' AS "TRUE";
-ERROR:  could not compute MD5 hash: unsupported
+ERROR:  could not compute MD5 hash: disabled for FIPS
 select md5('a') = '0cc175b9c0f1b6a831c399e269772661' AS "TRUE";
-ERROR:  could not compute MD5 hash: unsupported
+ERROR:  could not compute MD5 hash: disabled for FIPS
(etc etc)

Should we revisit the decision to not support this spelling
of the error message?  SLES 15 has got another decade or so
of support according to wikipedia [1], so it's hard to call it
a dead platform.

It looks like it'd be easy enough to generate the required
alternate expected-file, just s/unsupported/disabled for FIPS/g.
Happy to take care of this if there are not objections.

			regards, tom lane

[1] https://en.wikipedia.org/wiki/SUSE_Linux_Enterprise#End-of-support_schedule