Horton

Thread

Commits

  1. Use CFLAGS_SL while probing linkability of libperl.

  2. Select CFLAGS_SL at configure time, not in platform-specific Makefiles.

  1. configure fails for perl check on CentOS8

    Kyotaro Horiguchi <horikyota.ntt@gmail.com> — 2019-10-10T05:46:11Z

    Hello, While I'm moving to CentOS8 environment, I got stuck at
./configure with the following error.

configure: error: libperl library is requred for Perl

It complains that it needs -fPIC.

Configure uses only $Config{ccflags}, but it seems that
$Config{cccdlflags} is also required. The attached patch make
./configure success. (configure itself is excluded in the patch.)

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

  2. Re: configure fails for perl check on CentOS8

    Andrew Dunstan <andrew.dunstan@2ndquadrant.com> — 2019-10-10T13:22:48Z

    On 10/10/19 1:46 AM, Kyotaro Horiguchi wrote:
> Hello, While I'm moving to CentOS8 environment, I got stuck at
> ./configure with the following error.
>
> configure: error: libperl library is requred for Perl
>
> It complains that it needs -fPIC.
>
> Configure uses only $Config{ccflags}, but it seems that
> $Config{cccdlflags} is also required. The attached patch make
> ./configure success. (configure itself is excluded in the patch.)
>


./configure --with-perl


is working for me on Centos8 (double checked after a `dnf update`)


cheers


andrew


-- 

Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

  3. Re: configure fails for perl check on CentOS8

    Tom Lane <tgl@sss.pgh.pa.us> — 2019-10-10T15:51:21Z

    Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
> On 10/10/19 1:46 AM, Kyotaro Horiguchi wrote:
>> Hello, While I'm moving to CentOS8 environment, I got stuck at
>> ./configure with the following error.
>> configure: error: libperl library is requred for Perl
>> It complains that it needs -fPIC.
>> Configure uses only $Config{ccflags}, but it seems that
>> $Config{cccdlflags} is also required. The attached patch make
>> ./configure success. (configure itself is excluded in the patch.)

> ./configure --with-perl
> is working for me on Centos8 (double checked after a `dnf update`)

Yeah, I'm quite suspicious of this too.  Although we don't seem to have
any buildfarm members covering exactly RHEL8/CentOS8, we have enough
coverage of different Fedora releases to make it hard to believe that
we missed any changes in Red Hat's packaging of Perl.

Is this error perhaps occurring with a non-vendor Perl installation?
What's the exact error message?  config.log might contain some useful
clues, too.

			regards, tom lane

  4. Re: configure fails for perl check on CentOS8

    Kyotaro Horiguchi <horikyota.ntt@gmail.com> — 2019-10-15T10:45:34Z

    Hi. Sorry for the delay.

At Thu, 10 Oct 2019 11:51:21 -0400, Tom Lane <tgl@sss.pgh.pa.us> wrote in 
> Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
> > On 10/10/19 1:46 AM, Kyotaro Horiguchi wrote:
> >> Hello, While I'm moving to CentOS8 environment, I got stuck at
> >> ./configure with the following error.
> >> configure: error: libperl library is requred for Perl
> >> It complains that it needs -fPIC.
> >> Configure uses only $Config{ccflags}, but it seems that
> >> $Config{cccdlflags} is also required. The attached patch make
> >> ./configure success. (configure itself is excluded in the patch.)
> 
> > ./configure --with-perl
> > is working for me on Centos8 (double checked after a `dnf update`)
> 
> Yeah, I'm quite suspicious of this too.  Although we don't seem to have
> any buildfarm members covering exactly RHEL8/CentOS8, we have enough
> coverage of different Fedora releases to make it hard to believe that
> we missed any changes in Red Hat's packaging of Perl.
> 
> Is this error perhaps occurring with a non-vendor Perl installation?
> What's the exact error message?  config.log might contain some useful
> clues, too.

The perl package is official one. I found the cause, that's my
mistake.

The problematic command line boils down to:

$ ./configure --with-perl CFLAGS=-O0

It is bash-aliased and survived without being found for a long time on
my Cent7 environment, but CentOS8 doesn't allow that.

By the way, is there any official way to specify options like -O0
while configure time?

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

  5. Re: configure fails for perl check on CentOS8

    Andrew Dunstan <andrew.dunstan@2ndquadrant.com> — 2019-10-16T11:38:40Z

    On Tue, Oct 15, 2019 at 6:45 AM Kyotaro Horiguchi
<horikyota.ntt@gmail.com> wrote:

> >
> > Is this error perhaps occurring with a non-vendor Perl installation?
> > What's the exact error message?  config.log might contain some useful
> > clues, too.
>
> The perl package is official one. I found the cause, that's my
> mistake.
>
> The problematic command line boils down to:
>
> $ ./configure --with-perl CFLAGS=-O0
>
> It is bash-aliased and survived without being found for a long time on
> my Cent7 environment, but CentOS8 doesn't allow that.
>
> By the way, is there any official way to specify options like -O0
> while configure time?
>


CFLAGS=-O0 configure  --with-perl ...

cheers

andrew


-- 
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

  6. Re: configure fails for perl check on CentOS8

    Tom Lane <tgl@sss.pgh.pa.us> — 2019-10-16T12:32:48Z

    Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
> On Tue, Oct 15, 2019 at 6:45 AM Kyotaro Horiguchi
> <horikyota.ntt@gmail.com> wrote:
>> The problematic command line boils down to:
>> $ ./configure --with-perl CFLAGS=-O0
>> It is bash-aliased and survived without being found for a long time on
>> my Cent7 environment, but CentOS8 doesn't allow that.

I don't quite understand why that wouldn't work.

>> By the way, is there any official way to specify options like -O0
>> while configure time?

> CFLAGS=-O0 configure  --with-perl ...

The way Horiguchi-san did it has been supported by autoconf for
a good long time now, so I don't think command line syntax is
the issue.  Could that CFLAGS setting be interfering with some
feature test in our configure script?

			regards, tom lane

  7. Re: configure fails for perl check on CentOS8

    Andrew Dunstan <andrew.dunstan@2ndquadrant.com> — 2019-10-16T14:41:19Z

    On Wed, Oct 16, 2019 at 8:32 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
> > On Tue, Oct 15, 2019 at 6:45 AM Kyotaro Horiguchi
> > <horikyota.ntt@gmail.com> wrote:
> >> The problematic command line boils down to:
> >> $ ./configure --with-perl CFLAGS=-O0
> >> It is bash-aliased and survived without being found for a long time on
> >> my Cent7 environment, but CentOS8 doesn't allow that.
>
> I don't quite understand why that wouldn't work.
>
> >> By the way, is there any official way to specify options like -O0
> >> while configure time?
>
> > CFLAGS=-O0 configure  --with-perl ...
>
> The way Horiguchi-san did it has been supported by autoconf for
> a good long time now, so I don't think command line syntax is
> the issue.

Ah.

> Could that CFLAGS setting be interfering with some
> feature test in our configure script?
>
>


It looks like you need CFLAGS='-O0 -fPIC' on CentOS 8 when building with perl.

cheers

andrew



-- 
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

  8. Re: configure fails for perl check on CentOS8

    Kyotaro Horiguchi <horikyota.ntt@gmail.com> — 2019-10-17T03:39:33Z

    Hello.

# I'm almost comming back..

At Wed, 16 Oct 2019 10:41:19 -0400, Andrew Dunstan <andrew.dunstan@2ndquadrant.com> wrote in 
> On Wed, Oct 16, 2019 at 8:32 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > >> $ ./configure --with-perl CFLAGS=-O0
> > >> It is bash-aliased and survived without being found for a long time on
> > >> my Cent7 environment, but CentOS8 doesn't allow that.
> >
> > I don't quite understand why that wouldn't work.

I'm confused with make CFLAGS=..

> It looks like you need CFLAGS='-O0 -fPIC' on CentOS 8 when building with perl.

Yes, that is what my first mail said. I dug into it further.

The immediately problematic command generated by autoconf is:


gcc -o /tmp/conftest -Wall -Wmissing-prototypes -Wpointer-arith \
  -Wdeclaration-after-statement -Werror=vla -Wendif-labels \
  -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing \
  -fwrapv -fexcess-precision=standard -Wno-format-truncation \
  -Wno-stringop-truncation \
  -O0 \
  -D_GNU_SOURCE  -I/usr/lib64/perl5/CORE \
  /tmp/conftest.c \
  -Wl,-z,relro -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld \
  -fstack-protector-strong -L/usr/local/lib  -L/usr/lib64/perl5/CORE \
  -lperl -lpthread -lresolv -ldl -lm -lcrypt -lutil -lc

/usr/bin/ld: /tmp/ccGxodNv.o: relocation R_X86_64_32 against symbol `PL_memory_wrap' can not be used when making a PIE object; recompile with -fPIC
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status

Very interestingly I don't get the error when the "-O0" is "-O2". It
is because gcc eliminates the PL_memory_wrap maybe by inlining.

The *final* problematic command boils down to:

gcc -o /tmp/conftest /tmp/conftest_O0.o \
  -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -lperl

That is, the problem doesn't happen without
"-specs=/usr/lib/rpm/redhat/redhat-hardened-ld".

I found the following bug report but it hasn't ever been fixed since
2016 on Fedora 24. I'm not sure about the newer versions.

https://bugzilla.redhat.com/show_bug.cgi?id=1343892

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

  9. Re: configure fails for perl check on CentOS8

    Tom Lane <tgl@sss.pgh.pa.us> — 2019-10-18T13:50:31Z

    Kyotaro Horiguchi <horikyota.ntt@gmail.com> writes:
> The immediately problematic command generated by autoconf is:
> ...
> /usr/bin/ld: /tmp/ccGxodNv.o: relocation R_X86_64_32 against symbol `PL_memory_wrap' can not be used when making a PIE object; recompile with -fPIC
> /usr/bin/ld: final link failed: Nonrepresentable section on output
> collect2: error: ld returned 1 exit status

> Very interestingly I don't get the error when the "-O0" is "-O2". It
> is because gcc eliminates the PL_memory_wrap maybe by inlining.

Yeah, probably so.  But I don't like the idea of fixing a problem
triggered by user-supplied CFLAGS by injecting more cflags from
elsewhere.  That seems likely to be counterproductive, or at
least it risks overriding what the user wanted.

Can we fix this by using something other than perl_alloc() as
the tested-for function?  That is surely a pretty arbitrary
choice.  Are there any standard Perl entry points that are just
plain functions with no weird macro expansions?

			regards, tom lane

  10. Re: configure fails for perl check on CentOS8

    Andrew Dunstan <andrew.dunstan@2ndquadrant.com> — 2019-10-19T15:55:39Z

    On 10/18/19 9:50 AM, Tom Lane wrote:
> Kyotaro Horiguchi <horikyota.ntt@gmail.com> writes:
>> The immediately problematic command generated by autoconf is:
>> ...
>> /usr/bin/ld: /tmp/ccGxodNv.o: relocation R_X86_64_32 against symbol `PL_memory_wrap' can not be used when making a PIE object; recompile with -fPIC
>> /usr/bin/ld: final link failed: Nonrepresentable section on output
>> collect2: error: ld returned 1 exit status
>> Very interestingly I don't get the error when the "-O0" is "-O2". It
>> is because gcc eliminates the PL_memory_wrap maybe by inlining.
> Yeah, probably so.  But I don't like the idea of fixing a problem
> triggered by user-supplied CFLAGS by injecting more cflags from
> elsewhere.  That seems likely to be counterproductive, or at
> least it risks overriding what the user wanted.
>
> Can we fix this by using something other than perl_alloc() as
> the tested-for function?  That is surely a pretty arbitrary
> choice.  Are there any standard Perl entry points that are just
> plain functions with no weird macro expansions?
>


I had a look in perl's proto.h but didn't see any obvious candidate. I
tried a couple of others (e.g. Perl_get_context) and got the same result
reported above.


cheers


andrew


-- 
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

  11. Re: configure fails for perl check on CentOS8

    Tom Lane <tgl@sss.pgh.pa.us> — 2019-10-20T17:23:10Z

    Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
> On 10/18/19 9:50 AM, Tom Lane wrote:
>> Can we fix this by using something other than perl_alloc() as
>> the tested-for function?  That is surely a pretty arbitrary
>> choice.  Are there any standard Perl entry points that are just
>> plain functions with no weird macro expansions?

> I had a look in perl's proto.h but didn't see any obvious candidate. I
> tried a couple of others (e.g. Perl_get_context) and got the same result
> reported above.

I poked into this on a Fedora 30 installation and determined that the
stray reference is coming from this bit in Perl's inline.h:

/* saves machine code for a common noreturn idiom typically used in Newx*() */
GCC_DIAG_IGNORE_DECL(-Wunused-function);
static void
S_croak_memory_wrap(void)
{
    Perl_croak_nocontext("%s",PL_memory_wrap);
}
GCC_DIAG_RESTORE_DECL;

Apparently, gcc is smart enough to optimize this away as unused ...
at any optimization level higher than -O0.  I confirmed that it works
at -O0 too, if you change this function declaration to "static inline"
--- but evidently, not doing so was intentional, so we won't get much
cooperation if we propose changing it (back?) to a plain static inline.

So the failure occurs just from reading this header, independently of
which particular Perl function we try to call; I'd supposed that there
was some connection between perl_alloc and PL_memory_wrap, but there
isn't.

I still don't much like the originally proposed patch.  IMO it makes too
many assumptions about what is in Perl's ccflags, and perhaps more to the
point, it is testing libperl's linkability using switches we will not use
when we actually build plperl.  So I think there's a pretty high risk of
breaking other cases if we fix it that way.

The right way to fix it, likely, is to add CFLAGS_SL while performing this
particular autoconf test, as that would replicate the switches used for
plperl (and it turns out that adding -fPIC does solve this problem).
But the configure script doesn't currently know about CFLAGS_SL, so we'd
have to do some refactoring to approach it that way.  Moving that logic
from the platform-specific Makefiles into configure doesn't seem
unreasonable, but it would make the patch bigger.

A less invasive idea is to forcibly add -O1 to CFLAGS for this autoconf
test.  We'd have to be careful about doing so for non-gcc compilers, as
they might not understand that switch syntax ... but probably we could
get away with changing CFLAGS only when using a gcc-alike.  Still, that's
a hack, and it doesn't have much to recommend it other than being more
localized.

			regards, tom lane

  12. Re: configure fails for perl check on CentOS8

    Andrew Dunstan <andrew.dunstan@2ndquadrant.com> — 2019-10-20T19:31:06Z

    On 10/20/19 1:23 PM, Tom Lane wrote:
> Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
>> On 10/18/19 9:50 AM, Tom Lane wrote:
>>> Can we fix this by using something other than perl_alloc() as
>>> the tested-for function?  That is surely a pretty arbitrary
>>> choice.  Are there any standard Perl entry points that are just
>>> plain functions with no weird macro expansions?
>> I had a look in perl's proto.h but didn't see any obvious candidate. I
>> tried a couple of others (e.g. Perl_get_context) and got the same result
>> reported above.
> I poked into this on a Fedora 30 installation and determined that the
> stray reference is coming from this bit in Perl's inline.h:
>
> /* saves machine code for a common noreturn idiom typically used in Newx*() */
> GCC_DIAG_IGNORE_DECL(-Wunused-function);
> static void
> S_croak_memory_wrap(void)
> {
>     Perl_croak_nocontext("%s",PL_memory_wrap);
> }
> GCC_DIAG_RESTORE_DECL;
>
> Apparently, gcc is smart enough to optimize this away as unused ...
> at any optimization level higher than -O0.  I confirmed that it works
> at -O0 too, if you change this function declaration to "static inline"
> --- but evidently, not doing so was intentional, so we won't get much
> cooperation if we propose changing it (back?) to a plain static inline.
>
> So the failure occurs just from reading this header, independently of
> which particular Perl function we try to call; I'd supposed that there
> was some connection between perl_alloc and PL_memory_wrap, but there
> isn't.



Yeah, I came to the same conclusion.


>
> I still don't much like the originally proposed patch.  IMO it makes too
> many assumptions about what is in Perl's ccflags, and perhaps more to the
> point, it is testing libperl's linkability using switches we will not use
> when we actually build plperl.  So I think there's a pretty high risk of
> breaking other cases if we fix it that way.


Agreed.


>
> The right way to fix it, likely, is to add CFLAGS_SL while performing this
> particular autoconf test, as that would replicate the switches used for
> plperl (and it turns out that adding -fPIC does solve this problem).
> But the configure script doesn't currently know about CFLAGS_SL, so we'd
> have to do some refactoring to approach it that way.  Moving that logic
> from the platform-specific Makefiles into configure doesn't seem
> unreasonable, but it would make the patch bigger.


Sounds like a plan. I agree it's annoying to have to do something large
for something so trivial.


>
> A less invasive idea is to forcibly add -O1 to CFLAGS for this autoconf
> test.  We'd have to be careful about doing so for non-gcc compilers, as
> they might not understand that switch syntax ... but probably we could
> get away with changing CFLAGS only when using a gcc-alike.  Still, that's
> a hack, and it doesn't have much to recommend it other than being more
> localized.
>
> 			


right. I think your other plan is better.


cheers


andrew




-- 
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

  13. Re: configure fails for perl check on CentOS8

    Tom Lane <tgl@sss.pgh.pa.us> — 2019-10-20T23:36:39Z

    Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
> On 10/20/19 1:23 PM, Tom Lane wrote:
>> The right way to fix it, likely, is to add CFLAGS_SL while performing this
>> particular autoconf test, as that would replicate the switches used for
>> plperl (and it turns out that adding -fPIC does solve this problem).
>> But the configure script doesn't currently know about CFLAGS_SL, so we'd
>> have to do some refactoring to approach it that way.  Moving that logic
>> from the platform-specific Makefiles into configure doesn't seem
>> unreasonable, but it would make the patch bigger.

> Sounds like a plan. I agree it's annoying to have to do something large
> for something so trivial.

Turns out it's not really that bad.  We just have to transfer the
responsibility for setting CFLAGS_SL from the platform Makefiles
to the platform template files.  (As a bonus, it'd be possible to
allow users to override CFLAGS_SL during configure, as they can
do for CFLAGS.  But I didn't mess with that here.)

I checked that this fixes the Fedora build problem, but I've not
really tested it on any other platform.  Still, there's not that
much to go wrong, one would think.

			regards, tom lane

  14. Re: configure fails for perl check on CentOS8

    Andrew Dunstan <andrew.dunstan@2ndquadrant.com> — 2019-10-21T12:29:39Z

    On 10/20/19 7:36 PM, Tom Lane wrote:
> Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
>> On 10/20/19 1:23 PM, Tom Lane wrote:
>>> The right way to fix it, likely, is to add CFLAGS_SL while performing this
>>> particular autoconf test, as that would replicate the switches used for
>>> plperl (and it turns out that adding -fPIC does solve this problem).
>>> But the configure script doesn't currently know about CFLAGS_SL, so we'd
>>> have to do some refactoring to approach it that way.  Moving that logic
>>> from the platform-specific Makefiles into configure doesn't seem
>>> unreasonable, but it would make the patch bigger.
>> Sounds like a plan. I agree it's annoying to have to do something large
>> for something so trivial.
> Turns out it's not really that bad.  We just have to transfer the
> responsibility for setting CFLAGS_SL from the platform Makefiles
> to the platform template files.  (As a bonus, it'd be possible to
> allow users to override CFLAGS_SL during configure, as they can
> do for CFLAGS.  But I didn't mess with that here.)
>
> I checked that this fixes the Fedora build problem, but I've not
> really tested it on any other platform.  Still, there's not that
> much to go wrong, one would think.
>
> 			



LGTM


cheers


andrew


-- 
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

  15. Re: configure fails for perl check on CentOS8

    Kyotaro Horiguchi <horikyota.ntt@gmail.com> — 2019-10-23T06:04:10Z

    Hello.

At Mon, 21 Oct 2019 08:29:39 -0400, Andrew Dunstan <andrew.dunstan@2ndquadrant.com> wrote in 
> 
> On 10/20/19 7:36 PM, Tom Lane wrote:
> > Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
> >> On 10/20/19 1:23 PM, Tom Lane wrote:
> >>> The right way to fix it, likely, is to add CFLAGS_SL while performing this
> >>> particular autoconf test, as that would replicate the switches used for
> >>> plperl (and it turns out that adding -fPIC does solve this problem).
> >>> But the configure script doesn't currently know about CFLAGS_SL, so we'd
> >>> have to do some refactoring to approach it that way.  Moving that logic
> >>> from the platform-specific Makefiles into configure doesn't seem
> >>> unreasonable, but it would make the patch bigger.
> >> Sounds like a plan. I agree it's annoying to have to do something large
> >> for something so trivial.
> > Turns out it's not really that bad.  We just have to transfer the
> > responsibility for setting CFLAGS_SL from the platform Makefiles
> > to the platform template files.  (As a bonus, it'd be possible to
> > allow users to override CFLAGS_SL during configure, as they can
> > do for CFLAGS.  But I didn't mess with that here.)
> >
> > I checked that this fixes the Fedora build problem, but I've not
> > really tested it on any other platform.  Still, there's not that
> > much to go wrong, one would think.
> >
> 
> LGTM

However it's done, but it looks good to me and actually the problem is
gone. Thaks!

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center