Horton

Re: JIT compiling with LLVM v12.2

Andres Freund <andres@anarazel.de>

View thread View on postgr.es
From: Andres Freund <andres@anarazel.de>
To: Thomas Munro <thomas.munro@enterprisedb.com>
Cc: Stephen Frost <sfrost@snowman.net>, Pg Hackers <pgsql-hackers@postgresql.org>
Date: 2018-03-21T19:47:41Z
Lists: pgsql-hackers 
Hi,

On 2018-03-21 20:06:49 +1300, Thomas Munro wrote:
> On Wed, Mar 21, 2018 at 4:07 PM, Andres Freund <andres@anarazel.de> wrote:
> > Indeed. I've pushed a rebased version now, that basically just fixes the
> > issue Thomas observed.
> 
> I set up a 32 bit i386 virtual machine and installed Debian 9.4.
> Compiler warnings:

Was that with a 64bit CPU and 32bit OS, or actually a 32bit CPU?


> gcc -Wall -Wmissing-prototypes -Wpointer-arith
> -Wdeclaration-after-statement -Wendif-labels
> -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing
> -fwrapv -fexcess-precision=standard -g -O2  -fPIC
> -D__STDC_LIMIT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_CONSTANT_MACROS
> -D_GNU_SOURCE -I/usr/lib/llvm-3.9/include  -I../../../../src/include
> -D_GNU_SOURCE   -c -o llvmjit.o llvmjit.c
> llvmjit.c: In function ‘llvm_get_function’:
> llvmjit.c:268:10: warning: cast to pointer from integer of different
> size [-Wint-to-pointer-cast]
>    return (void *) addr;
>           ^
> llvmjit.c:270:10: warning: cast to pointer from integer of different
> size [-Wint-to-pointer-cast]
>    return (void *) addr;
>           ^
> llvmjit.c: In function ‘llvm_resolve_symbol’:
> llvmjit.c:842:10: warning: cast from pointer to integer of different
> size [-Wpointer-to-int-cast]
>    addr = (uint64_t) load_external_function(modname, funcname,
>           ^
> llvmjit.c:845:10: warning: cast from pointer to integer of different
> size [-Wpointer-to-int-cast]
>    addr = (uint64_t) LLVMSearchForAddressOfSymbol(symname);
>           ^

Hrmpf, those need to be fixed.



> While trying out many combinations of versions of stuff on different
> OSes, I found another way to screw up that I wanted to report here.
> It's obvious that this is doomed if you know what's going on, but I
> thought the failure mode was interesting enough to report here.  There
> is a hazard for people running systems where the vendor ships some
> version (possibly a mystery version) of clang in the PATH but you have
> to get LLVM separately (eg from ports/brew/whatever):

> 1.  If you use macOS High Sierra's current /usr/bin/clang ("9.0.0"),
> ie the default if you didn't set CLANG to something else when you ran
> ./configure, and you build against LLVM 3.9, then llvm-lto gives this
> message during "make install":
> 
> Invalid summary version 3, 1 expected
> error: can't create ModuleSummaryIndexObjectFile for buffer: Corrupted bitcode
> 
> Then it segfaults!

Gah, that's not desirable :/.  It's fine that it doesn't work, but it'd
be better if it didn't segfault. I guess I could just try by corrupting
the file explicitly...



> Neither of these cases should be too surprising, and users of those
> operating systems can easily get a newer LLVM or an older -- it was
> just interesting to see exactly what goes wrong and exactly when.  I
> suppose there could be a configure test to see if your $CLANG can play
> nicely with your $LLVM_CONFIG.

Not precisely sure how.  I think suggesting to use compatible clang is
going to be sufficient for most cases...

Greetings,

Andres Freund

Commits

  1. Improve JIT docs.

  2. Add documentation for the JIT feature.

  3. Add EXPLAIN support for JIT.

  4. Add inlining support to LLVM JIT provider.

  5. JIT tuple deforming in LLVM JIT provider.

  6. Add FIELDNO_* macro designating offset into structs required for JIT.

  7. Add expression compilation support to LLVM JIT provider.

  8. Expand list of synchronized types and functions in LLVM JIT provider.

  9. Add helpers for emitting LLVM IR.

  10. Basic planner and executor integration for JIT.

  11. Debugging and profiling support for LLVM JIT provider.

  12. Support for optimizing and emitting code in LLVM JIT provider.

  13. Add file containing extensions of the LLVM C API.

  14. Basic JIT provider and error handling infrastructure.

  15. Add configure infrastructure (--with-llvm) to enable LLVM support.

  16. Add C++ support to configure.

  17. Add PGAC_PROG_VARCC_VARFLAGS_OPT autoconf macro.

  18. Fix VM buffer pin management in heap_lock_updated_tuple_rec().

  19. Allow tupleslots to have a fixed tupledesc, use in executor nodes.

  20. Expression evaluation based aggregate transition invocation.

  21. Perform slot validity checks in a separate pass over expression.

  22. Rely on executor utils to build targetlist for DML RETURNING.

  23. Refer to OS X as "macOS", except for the port name which is still "darwin".