Horton

Thread

Commits

  1. jit: configure: Explicitly reference 'native' component.

  1. PG vs LLVM 12 on seawasp, next round

    Thomas Munro <thomas.munro@gmail.com> — 2020-12-01T04:35:49Z

    Hi

Since seawasp's bleeding edge LLVM installation moved to "trunk
20201114 c8f4e06b 12.0.0" ~16 days ago, it has been red.  Further
updates didn't help it and it's now on "trunk 20201127 6ee22ca6
12.0.0".  I wonder if there is something in Fabien's scripting that
needs to be tweaked, perhaps a symlink name or similar.  I don't
follow LLVM development but I found my way to a commit[1] around the
right time that mentions breaking up the OrcJIT library, so *shrug*
maybe that's a clue.

+ERROR:  could not load library
"/home/fabien/pg/build-farm-11/buildroot/HEAD/pgsql.build/tmp_install/home/fabien/pg/build-farm-11/buildroot/HEAD/inst/lib/postgresql/llvmjit.so":
libLLVMOrcJIT.so.12git: cannot open shared object file: No such file
or directory

[1] https://github.com/llvm/llvm-project/commit/1d0676b54c4e3a517719220def96dfdbc26d8048

  2. Re: PG vs LLVM 12 on seawasp, next round

    Andres Freund <andres@anarazel.de> — 2020-12-01T06:49:49Z

    Hi,

On 2020-12-01 17:35:49 +1300, Thomas Munro wrote:
> Since seawasp's bleeding edge LLVM installation moved to "trunk
> 20201114 c8f4e06b 12.0.0" ~16 days ago, it has been red.  Further
> updates didn't help it and it's now on "trunk 20201127 6ee22ca6
> 12.0.0".  I wonder if there is something in Fabien's scripting that
> needs to be tweaked, perhaps a symlink name or similar.  I don't
> follow LLVM development but I found my way to a commit[1] around the
> right time that mentions breaking up the OrcJIT library, so *shrug*
> maybe that's a clue.
> 
> +ERROR:  could not load library
> "/home/fabien/pg/build-farm-11/buildroot/HEAD/pgsql.build/tmp_install/home/fabien/pg/build-farm-11/buildroot/HEAD/inst/lib/postgresql/llvmjit.so":
> libLLVMOrcJIT.so.12git: cannot open shared object file: No such file
> or directory

It's a change in how LLVM dependencies are declared
internally. Previously the 'native' component was - unintentionally -
transitively included via the 'orcjit' component, but now that's not the
case anymore.

The attached patch should fix it, I think?

Greetings,

Andres Freund

  3. Re: PG vs LLVM 12 on seawasp, next round

    Fabien COELHO <coelho@cri.ensmp.fr> — 2020-12-01T20:04:44Z

    Hello Thomas,

> Since seawasp's bleeding edge LLVM installation moved to "trunk
> 20201114 c8f4e06b 12.0.0" ~16 days ago, it has been red.  Further
> updates didn't help it and it's now on "trunk 20201127 6ee22ca6
> 12.0.0".  I wonder if there is something in Fabien's scripting that
> needs to be tweaked, perhaps a symlink name or similar.

The compiler compilation script is quite straightforward (basically, get 
sources, configure and compile), even for a such a moving target…

The right approach is to wait for some time before looking at the issue, 
typically one week for the next recompilation, in case the problem 
evaporates, so you were right not to jump on it right away:-)

Andres investigated a few days ago, managed to reproduce the issue 
locally, and has one line patch. I'm unsure if it should be prevently 
back-patched, though.

-- 
Fabien.

  4. Re: PG vs LLVM 12 on seawasp, next round

    Andres Freund <andres@anarazel.de> — 2020-12-01T20:08:10Z

    Hi,

On 2020-12-01 21:04:44 +0100, Fabien COELHO wrote:
> Andres investigated a few days ago, managed to reproduce the issue locally,
> and has one line patch. I'm unsure if it should be prevently back-patched,
> though.

I see no reason not to backpatch - it's more correct for past versions
of LLVM as well.

Greetings,

Andres Freund

  5. Re: PG vs LLVM 12 on seawasp, next round

    Andres Freund <andres@anarazel.de> — 2020-12-08T03:38:19Z

    Hi,

On 2020-12-01 12:08:10 -0800, Andres Freund wrote:
> On 2020-12-01 21:04:44 +0100, Fabien COELHO wrote:
> > Andres investigated a few days ago, managed to reproduce the issue locally,
> > and has one line patch. I'm unsure if it should be prevently back-patched,
> > though.
> 
> I see no reason not to backpatch - it's more correct for past versions
> of LLVM as well.

I pushed that now.

- Andres

  6. Re: PG vs LLVM 12 on seawasp, next round

    Andres Freund <andres@anarazel.de> — 2020-12-08T06:35:51Z

    Hi,

On 2020-12-07 19:38:19 -0800, Andres Freund wrote:
> On 2020-12-01 12:08:10 -0800, Andres Freund wrote:
> > On 2020-12-01 21:04:44 +0100, Fabien COELHO wrote:
> > > Andres investigated a few days ago, managed to reproduce the issue locally,
> > > and has one line patch. I'm unsure if it should be prevently back-patched,
> > > though.
> > 
> > I see no reason not to backpatch - it's more correct for past versions
> > of LLVM as well.
> 
> I pushed that now.

I hadn't checked that before, but for the last few days there's been a
different failure than the one I saw earlier:

+ERROR:  could not load library "/home/fabien/pg/build-farm-11/buildroot/HEAD/pgsql.build/tmp_install/home/fabien/pg/build-farm-11/buildroot/HEAD/inst/lib/postgresql/llvmjit.so": libLLVMOrcJIT.so.12git: cannot open shared object file: No such file or directory

whereas what I fixed is about:

+ERROR:  could not load library "/home/fabien/pg/build-farm-11/buildroot/HEAD/pgsql.build/tmp_install/home/fabien/pg/build-farm-11/buildroot/HEAD/inst/lib/postgresql/llvmjit.so": /home/fabien/pg/build-farm-11/buildroot/HEAD/pgsql.build/tmp_install/home/fabien/pg/build-farm-11/buildroot/HEAD/inst/lib/postgresql/llvmjit.so: undefined symbol: LLVMInitializeX86Target

Changed somewhere between
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=seawasp&dt=2020-11-20%2009%3A17%3A10
and
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=seawasp&dt=2020-11-21%2023%3A17%3A11

The "no such file" error seems more like a machine local issue to me.

Greetings,

Andres Freund

  7. Re: PG vs LLVM 12 on seawasp, next round

    Fabien COELHO <coelho@cri.ensmp.fr> — 2020-12-11T19:45:12Z

    Hello Andres,

> I hadn't checked that before, but for the last few days there's been a
> different failure than the one I saw earlier:
>
> +ERROR:  could not load library "/home/fabien/pg/build-farm-11/buildroot/HEAD/pgsql.build/tmp_install/home/fabien/pg/build-farm-11/buildroot/HEAD/inst/lib/postgresql/llvmjit.so": libLLVMOrcJIT.so.12git: cannot open shared object file: No such file or directory
>
> whereas what I fixed is about:
>
> +ERROR:  could not load library "/home/fabien/pg/build-farm-11/buildroot/HEAD/pgsql.build/tmp_install/home/fabien/pg/build-farm-11/buildroot/HEAD/inst/lib/postgresql/llvmjit.so": /home/fabien/pg/build-farm-11/buildroot/HEAD/pgsql.build/tmp_install/home/fabien/pg/build-farm-11/buildroot/HEAD/inst/lib/postgresql/llvmjit.so: undefined symbol: LLVMInitializeX86Target
>
> Changed somewhere between
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=seawasp&dt=2020-11-20%2009%3A17%3A10
> and
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=seawasp&dt=2020-11-21%2023%3A17%3A11
>
> The "no such file" error seems more like a machine local issue to me.

I'll look into it when I have time, which make take some time. Hopefully 
over the holidays.

-- 
Fabien.

  8. Re: PG vs LLVM 12 on seawasp, next round

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2021-01-15T17:06:31Z

    On 2020-Dec-11, Fabien COELHO wrote:

> > I hadn't checked that before, but for the last few days there's been a
> > different failure than the one I saw earlier:
> > 
> > +ERROR:  could not load library "/home/fabien/pg/build-farm-11/buildroot/HEAD/pgsql.build/tmp_install/home/fabien/pg/build-farm-11/buildroot/HEAD/inst/lib/postgresql/llvmjit.so": libLLVMOrcJIT.so.12git: cannot open shared object file: No such file or directory

> > The "no such file" error seems more like a machine local issue to me.
> 
> I'll look into it when I have time, which make take some time. Hopefully
> over the holidays.

This is still happening ... Any chance you can have a look at it?

-- 
Álvaro Herrera                            39°49'30"S 73°17'W

  9. Re: PG vs LLVM 12 on seawasp, next round

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-01-15T17:43:05Z

    Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> On 2020-Dec-11, Fabien COELHO wrote:
>> I'll look into it when I have time, which make take some time. Hopefully
>> over the holidays.

> This is still happening ... Any chance you can have a look at it?

If you don't have time to debug it, perhaps you could just disable
the buildfarm animal till you do.  It's cluttering the buildfarm
failure report without providing useful info ...

			regards, tom lane

  10. Re: PG vs LLVM 12 on seawasp, next round

    Fabien COELHO <coelho@cri.ensmp.fr> — 2021-01-18T20:29:53Z

    Hello Alvaro,

>>> The "no such file" error seems more like a machine local issue to me.
>>
>> I'll look into it when I have time, which make take some time. Hopefully
>> over the holidays.
>
> This is still happening ... Any chance you can have a look at it?

Indeed. I'll try to look (again) into it soon. I had a look but did not 
find anything obvious in the short time frame I had. Last two months were 
a little overworked for me so I let slip quite a few things. If you want 
to disable the animal as Tom suggests, do as you want.

-- 
Fabien.

  11. Re: PG vs LLVM 12 on seawasp, next round

    Noah Misch <noah@leadboat.com> — 2021-01-25T03:40:51Z

    On Mon, Jan 18, 2021 at 09:29:53PM +0100, Fabien COELHO wrote:
> >>>The "no such file" error seems more like a machine local issue to me.
> >>
> >>I'll look into it when I have time, which make take some time. Hopefully
> >>over the holidays.
> >
> >This is still happening ... Any chance you can have a look at it?
> 
> Indeed. I'll try to look (again) into it soon. I had a look but did not find
> anything obvious in the short time frame I had. Last two months were a
> little overworked for me so I let slip quite a few things. If you want to
> disable the animal as Tom suggests, do as you want.

Perhaps he was suggesting that you (buildfarm owner) disable the cron job that
initiates new runs.  That's what I do when one of my animals needs my
intervention.

  12. Re: PG vs LLVM 12 on seawasp, next round

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-01-25T04:06:18Z

    Noah Misch <noah@leadboat.com> writes:
> On Mon, Jan 18, 2021 at 09:29:53PM +0100, Fabien COELHO wrote:
>> ... Last two months were a
>> little overworked for me so I let slip quite a few things. If you want to
>> disable the animal as Tom suggests, do as you want.

> Perhaps he was suggesting that you (buildfarm owner) disable the cron job that
> initiates new runs.  That's what I do when one of my animals needs my
> intervention.

Indeed.  I'm not sure there even is a provision to block an animal on the
buildfarm-server side.  If there is, you'd have to request that it be
manually undone after you get around to fixing the animal.  Frankly,
if I were the BF admin, I would be in about as much hurry to do that
as you've been to fix it.

			regards, tom lane

  13. Re: PG vs LLVM 12 on seawasp, next round

    Thomas Munro <thomas.munro@gmail.com> — 2021-02-15T05:32:46Z

    On Sat, Dec 12, 2020 at 8:45 AM Fabien COELHO <coelho@cri.ensmp.fr> wrote:
> > +ERROR:  could not load library "/home/fabien/pg/build-farm-11/buildroot/HEAD/pgsql.build/tmp_install/home/fabien/pg/build-farm-11/buildroot/HEAD/inst/lib/postgresql/llvmjit.so": libLLVMOrcJIT.so.12git: cannot open shared object file: No such file or directory

Bonjour Fabien,

Here is the creation of llvmjit.so:

g++ ... -o llvmjit.so ... -L/home/fabien/clgtk/lib ... -lLLVMOrcJIT ...

That'd be from llvm-config --ldflags or similar, from this binary:

checking for llvm-config... (cached) /home/fabien/clgtk/bin/llvm-config

So what does ls -slap /home/fabien/clgtk/lib show?  What does ldd
/home/fabien/pg/build-farm-11/buildroot/HEAD/pgsql.build/tmp_install/home/fabien/pg/build-farm-11/buildroot/HEAD/inst/lib/postgresql/llvmjit.so
show?  What do you have in seawap's LD_LIBRARY_PATH, and is the
problem that you need to add /home/fabien/clgtk/lib to it (or is it
supposed to be making it into the rpath, or is it in your ld.so.conf)?

PS Could you try blowing away the accache directory so we can rule out
bad cached configure stuff?

  14. Re: PG vs LLVM 12 on seawasp, next round

    Fabien COELHO <coelho@cri.ensmp.fr> — 2021-02-15T09:05:32Z

    Hello Thomas,

Thanks for looking at this, I'm currently far behind on many things and 
not very responsive:-(

> Here is the creation of llvmjit.so:
>
> g++ ... -o llvmjit.so ... -L/home/fabien/clgtk/lib ... -lLLVMOrcJIT ...
>
> That'd be from llvm-config --ldflags or similar, from this binary:
>
> checking for llvm-config... (cached) /home/fabien/clgtk/bin/llvm-config
>
> So what does ls -slap /home/fabien/clgtk/lib show?

Plenty files and links, eg:

      0 lrwxrwxrwx 1 fabien fabien       21 janv. 23 09:40 /home/fabien/clgtk/lib/libLLVMMCJIT.so -> libLLVMMCJIT.so.12git
   2140 -rw-r--r-- 1 fabien fabien  2190824 janv. 23 09:28 /home/fabien/clgtk/lib/libLLVMMCJIT.so.12git
      0 lrwxrwxrwx 1 fabien fabien       22 janv. 23 09:40 /home/fabien/clgtk/lib/libLLVMOrcJIT.so -> libLLVMOrcJIT.so.12git
  40224 -rw-r--r-- 1 fabien fabien 41187208 janv. 23 09:34 /home/fabien/clgtk/lib/libLLVMOrcJIT.so.12git

Hmmm, clang recompilation has been failing for some weeks. Argh, this is 
due to the recent "master" to "main" branch renaming.

>  What does ldd
> /home/fabien/pg/build-farm-11/buildroot/HEAD/pgsql.build/tmp_install/home/fabien/pg/build-farm-11/buildroot/HEAD/inst/lib/postgresql/llvmjit.so

No such file of directory:-) because the directory is cleaned up.

> show?  What do you have in seawap's LD_LIBRARY_PATH, and is the
> problem that you need to add /home/fabien/clgtk/lib to it

Argh. Would it be so stupid? :-( I thought the configuration stuff would
manage the link path automatically, which may be quite naïve, indeed.

> PS Could you try blowing away the accache directory so we can rule out
> bad cached configure stuff?

Hmmm. I've tried that before. I can do it again.

I've added an explicit LD_LIBRARY_PATH, which will be triggered at some 
point later.

-- 
Fabien Coelho - CRI, MINES ParisTech

  15. Re: PG vs LLVM 12 on seawasp, next round

    Andres Freund <andres@anarazel.de> — 2021-02-15T18:01:58Z

    Hi,

On 2021-02-15 10:05:32 +0100, Fabien COELHO wrote:
> > show?  What do you have in seawap's LD_LIBRARY_PATH, and is the
> > problem that you need to add /home/fabien/clgtk/lib to it
> 
> Argh. Would it be so stupid? :-( I thought the configuration stuff would
> manage the link path automatically, which may be quite naïve, indeed.

The only way to do that is to add an rpath annotation, and e.g. distros
don't like that - there's some security implications.

> I've added an explicit LD_LIBRARY_PATH, which will be triggered at some
> point later.

You can also do something like LDFLAGS="$LDFLAGS -Wl,-rpath,$(llvm-config --libdir)"
or such.

Greetings,

Andres Freund

  16. Re: PG vs LLVM 12 on seawasp, next round

    Fabien COELHO <coelho@cri.ensmp.fr> — 2021-02-15T19:41:39Z

    >> I've added an explicit LD_LIBRARY_PATH, which will be triggered at some
>> point later.

This seems to have fixed the issue.

I'm sorry for the noise and quite baffled anyway, because according to my 
change logs it does not seem that I modified anything from my side about 
the dynamic library path when compiling with clang. At least I do not see 
a trace of that.

> You can also do something like LDFLAGS="$LDFLAGS -Wl,-rpath,$(llvm-config --libdir)"
> or such.

I've resorted to just hardcode LD_LIBRARY_PATH alongside PATH when 
compiling with clang in my buildfarm script. Thanks for the tip anyway.

And thanks Thomas for pointing out the fix!

-- 
Fabien.

  17. Re: PG vs LLVM 12 on seawasp, next round

    Andrew Dunstan <andrew@dunslane.net> — 2021-02-15T19:50:34Z

    On 1/24/21 11:06 PM, Tom Lane wrote:
> Noah Misch <noah@leadboat.com> writes:
>> On Mon, Jan 18, 2021 at 09:29:53PM +0100, Fabien COELHO wrote:
>>> ... Last two months were a
>>> little overworked for me so I let slip quite a few things. If you want to
>>> disable the animal as Tom suggests, do as you want.
>> Perhaps he was suggesting that you (buildfarm owner) disable the cron job that
>> initiates new runs.  That's what I do when one of my animals needs my
>> intervention.
> Indeed.  I'm not sure there even is a provision to block an animal on the
> buildfarm-server side.  If there is, you'd have to request that it be
> manually undone after you get around to fixing the animal.  Frankly,
> if I were the BF admin, I would be in about as much hurry to do that
> as you've been to fix it.
>
> 			


It's actually very easy, but it's something I usually reserve for people
who are very unresponsive to emails. As noted, disabling the crontab
entry on the client side is a preferable solution.


cheers


andrew


--
Andrew Dunstan
EDB: https://www.enterprisedb.com