Horton

Thread

Commits

  1. Be more paranoid in configure's checks for CRC and POPCNT intrinsics.

  1. Not-terribly-safe checks for CRC intrinsic support

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-03-14T23:04:22Z

    I noticed that our configuration-time checks for the presence
of CRC intrinsics generally look like

    unsigned int crc = 0;
    crc = __crc32cb(crc, 0);
    crc = __crc32ch(crc, 0);
    crc = __crc32cw(crc, 0);
    crc = __crc32cd(crc, 0);
    /* return computed value, to prevent the above being optimized away */
    return crc == 0;

The trouble with this is that "crc" is a local variable, so the
compiler would be perfectly within its rights to optimize the whole
thing down to "return some_constant".  While that outcome sufficiently
proves that the compiler has heard of these intrinsics, it fails to
prove that the platform has any necessary library infrastructure,
assembler support for the opcodes, etc etc.  Whoever originally
wrote this evidently had concern for that hazard, or they'd not
have bothered with forcing a dependency on the final value; but
that seems insufficient.  We have other nearby tests that try
to avoid this problem by making the functions-under-test operate
on global variables, so I think we should do likewise here.

In connection with bug #18839[1], I checked to see if this might
already be happening.  At least with gcc 12.2 on armhf Debian,
it doesn't seem to: the compiler still generates the crc opcodes.
But the same compiler is perfectly willing to optimize a call to
sin(3) down to a constant under similar conditions.  So I think this
is just a matter of they didn't get round to it, not that there's a
principled reason to think they won't ever get round to it.  There
might be other cases where these probes are already missing something,
and we've not noticed because there's-compiler-support-but-no-
library-support is surely a very rare case in the field.

In short, I think we ought to apply and perhaps back-patch something
like the attached.

BTW, it looks to me like PGAC_AVX512_POPCNT_INTRINSICS is at similar
hazard, but I'm not entirely sure how to fix that one.

Thoughts?

			regards, tom lane

[1] https://www.postgresql.org/message-id/18839-7615d0f8267dc015%40postgresql.org

  2. Re: Not-terribly-safe checks for CRC intrinsic support

    Steven Niu <niushiji@gmail.com> — 2025-03-17T01:43:26Z

    +# is missing, we must link not just compile, and store the results in 
global

The "compile" should be "compiler"?

Regards,
Steven

在 2025/3/15 7:04, Tom Lane 写道:
> I noticed that our configuration-time checks for the presence
> of CRC intrinsics generally look like
> 
>      unsigned int crc = 0;
>      crc = __crc32cb(crc, 0);
>      crc = __crc32ch(crc, 0);
>      crc = __crc32cw(crc, 0);
>      crc = __crc32cd(crc, 0);
>      /* return computed value, to prevent the above being optimized away */
>      return crc == 0;
> 
> The trouble with this is that "crc" is a local variable, so the
> compiler would be perfectly within its rights to optimize the whole
> thing down to "return some_constant".  While that outcome sufficiently
> proves that the compiler has heard of these intrinsics, it fails to
> prove that the platform has any necessary library infrastructure,
> assembler support for the opcodes, etc etc.  Whoever originally
> wrote this evidently had concern for that hazard, or they'd not
> have bothered with forcing a dependency on the final value; but
> that seems insufficient.  We have other nearby tests that try
> to avoid this problem by making the functions-under-test operate
> on global variables, so I think we should do likewise here.
> 
> In connection with bug #18839[1], I checked to see if this might
> already be happening.  At least with gcc 12.2 on armhf Debian,
> it doesn't seem to: the compiler still generates the crc opcodes.
> But the same compiler is perfectly willing to optimize a call to
> sin(3) down to a constant under similar conditions.  So I think this
> is just a matter of they didn't get round to it, not that there's a
> principled reason to think they won't ever get round to it.  There
> might be other cases where these probes are already missing something,
> and we've not noticed because there's-compiler-support-but-no-
> library-support is surely a very rare case in the field.
> 
> In short, I think we ought to apply and perhaps back-patch something
> like the attached.
> 
> BTW, it looks to me like PGAC_AVX512_POPCNT_INTRINSICS is at similar
> hazard, but I'm not entirely sure how to fix that one.
> 
> Thoughts?
> 
> 			regards, tom lane
> 
> [1] https://www.postgresql.org/message-id/18839-7615d0f8267dc015%40postgresql.org
>

  3. Re: Not-terribly-safe checks for CRC intrinsic support

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-03-17T01:56:19Z

    Steven Niu <niushiji@gmail.com> writes:
> +# is missing, we must link not just compile, and store the results in 
> global

> The "compile" should be "compiler"?

I think it's okay as-is: "link" and "compile" are both being used
as verbs.  We could say "run the compiler", but that's longer
without being better.

Besides which, I stole this comment verbatim from elsewhere
in the same file ;-)

			regards, tom lane

  4. Re: Not-terribly-safe checks for CRC intrinsic support

    David G. Johnston <david.g.johnston@gmail.com> — 2025-03-17T02:01:49Z

    On Sunday, March 16, 2025, Steven Niu <niushiji@gmail.com> wrote:

>
> +# is missing, we must link not just compile, and store the results in
> global
>
> The "compile" should be "compiler"?
>

No.  Compile is the verb that pairs with link.  Compiler is a noun, its
compliment being the linker.

I’d probably add a comma before the “not” though.  Or maybe: we must also
link and store the results in global

Doesn’t link imply compilation?

David J.

  5. Re: Not-terribly-safe checks for CRC intrinsic support

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-03-17T02:13:01Z

    "David G. Johnston" <david.g.johnston@gmail.com> writes:
> On Sunday, March 16, 2025, Steven Niu <niushiji@gmail.com> wrote:
>> +# is missing, we must link not just compile, and store the results in
>> global

> I’d probably add a comma before the “not” though.  Or maybe: we must also
> link and store the results in global

A comma there wouldn't be wrong, but in context that would make for
an overabundance of commas.  Or so it seems to me anyway.

> Doesn’t link imply compilation?

Yes.

			regards, tom lane

  6. Re: Not-terribly-safe checks for CRC intrinsic support

    John Naylor <johncnaylorls@gmail.com> — 2025-03-17T09:44:21Z

    On Sat, Mar 15, 2025 at 6:04 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> In short, I think we ought to apply and perhaps back-patch something
> like the attached.

Seems like reasonable defensive coding and consistency.

-    /* return computed value, to prevent the above being optimized away */
+    /* else this function could get optimized away altogether: */

-    /* return computed value, to prevent the above being optimized away */
+    /* return computed value, just to be extra sure this isn't
optimized away */

I'd be okay with keeping the original comment, though, since it seems
to be explaining the choice well enough.

> BTW, it looks to me like PGAC_AVX512_POPCNT_INTRINSICS is at similar
> hazard, but I'm not entirely sure how to fix that one.

"buf" is the variable there that we're loading from, so that would be
the one to make global.

-- 
John Naylor
Amazon Web Services

  7. Re: Not-terribly-safe checks for CRC intrinsic support

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-03-20T20:27:13Z

    John Naylor <johncnaylorls@gmail.com> writes:
> On Sat, Mar 15, 2025 at 6:04 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> In short, I think we ought to apply and perhaps back-patch something
>> like the attached.

> Seems like reasonable defensive coding and consistency.

Thanks for looking at it.

> I'd be okay with keeping the original comment, though, since it seems
> to be explaining the choice well enough.

Okay.

>> BTW, it looks to me like PGAC_AVX512_POPCNT_INTRINSICS is at similar
>> hazard, but I'm not entirely sure how to fix that one.

> "buf" is the variable there that we're loading from, so that would be
> the one to make global.

Ah.  I was confused by the "const" decoration, but we can remove that.

After thinking for a bit, I pushed this just to master rather than
back-patching.  We can do a back-patch if anyone discovers that this
is a live issue on any current platform, but I rather suspect that
it isn't.  Compiler not matched to platform is a situation that's
gone away for most people.

			regards, tom lane