Horton

Re: JIT compiling with LLVM v9.1

Andreas Karlsson <andreas@proxel.se>

View thread View on postgr.es
From: Andreas Karlsson <andreas@proxel.se>
To: Pierre Ducroquet <p.psql@pinaraf.info>, pgsql-hackers@lists.postgresql.org
Cc: Pierre Ducroquet <pierre.ducroquet@people-doc.com>, Andres Freund <andres@anarazel.de>
Date: 2018-02-15T11:54:34Z
Lists: pgsql-hackers 
On 02/05/2018 10:44 PM, Pierre Ducroquet wrote:
>> psqlscanslash.l: In function ‘psql_scan_slash_option’:
>> psqlscanslash.l:550:8: warning: variable ‘lexresult’ set but not used
>> [-Wunused-but-set-variable]
>>     int   final_state;
>>           ^~~~~~~~~
> 
> I'm not sure Andres's patches have anything to do with psql, it's surprising.

I managed to track down the bug and apparently when building with 
--with-llvm the -DNDEBUG option is added to CPPFLAGS, but I am not 
entirely sure what the code in config/llvm.m4 is trying to do in the 
first place.

The two issues I see with what the code does are:

1) Why does config/llvm.m4 modify CPPFLAGS? That affects the building of 
the binaries too which may be done with gcc like in my case. Shouldn't 
it use a LLVM_CPPFLAGS or something?

2) When I build with --with-cassert I expect the assertions to be there, 
both in the binaries and the bitcode. Is that just a bug or is there any 
thought behind this?

Below is the diff in src/Makefile.global between when I run configure 
with --with-llvm or not.

diff src/Makefile.global-nollvm  src/Makefile.global-llvm
78c78
< configure_args =  '--prefix=/home/andreas/dev/postgresql-inst' 
'--enable-tap-tests' '--enable-cassert' '--enable-debug'
---
 > configure_args =  '--prefix=/home/andreas/dev/postgresql-inst' 
'--enable-tap-tests' '--enable-cassert' '--enable-debug' '--with-llvm'
190c190
< with_llvm	= no
---
 > with_llvm	= yes
227,229c227,229
< LLVM_CONFIG =
< LLVM_BINPATH =
< CLANG =
---
 > LLVM_CONFIG = /usr/bin/llvm-config
 > LLVM_BINPATH = /usr/lib/llvm-4.0/bin
 > CLANG = /usr/bin/clang
238c238
< CPPFLAGS =  -D_GNU_SOURCE
---
 > CPPFLAGS = -D__STDC_LIMIT_MACROS -D__STDC_FORMAT_MACROS 
-D__STDC_CONSTANT_MACROS -D_GNU_SOURCE -DNDEBUG 
-I/usr/lib/llvm-4.0/include  -D_GNU_SOURCE
261c261
< LLVM_CXXFLAGS =
---
 > LLVM_CXXFLAGS =  -std=c++0x -std=c++11 -fno-exceptions
283c283
< LLVM_LIBS=
---
 > LLVM_LIBS= -lLLVM-4.0
297c297
< LDFLAGS +=   -Wl,--as-needed
---
 > LDFLAGS +=  -L/usr/lib/llvm-4.0/lib  -Wl,--as-needed

Commits

  1. Improve JIT docs.

  2. Add documentation for the JIT feature.

  3. Add EXPLAIN support for JIT.

  4. Add inlining support to LLVM JIT provider.

  5. JIT tuple deforming in LLVM JIT provider.

  6. Add FIELDNO_* macro designating offset into structs required for JIT.

  7. Add expression compilation support to LLVM JIT provider.

  8. Expand list of synchronized types and functions in LLVM JIT provider.

  9. Add helpers for emitting LLVM IR.

  10. Basic planner and executor integration for JIT.

  11. Debugging and profiling support for LLVM JIT provider.

  12. Support for optimizing and emitting code in LLVM JIT provider.

  13. Add file containing extensions of the LLVM C API.

  14. Basic JIT provider and error handling infrastructure.

  15. Add configure infrastructure (--with-llvm) to enable LLVM support.

  16. Add C++ support to configure.

  17. Add PGAC_PROG_VARCC_VARFLAGS_OPT autoconf macro.

  18. Fix VM buffer pin management in heap_lock_updated_tuple_rec().

  19. Allow tupleslots to have a fixed tupledesc, use in executor nodes.

  20. Expression evaluation based aggregate transition invocation.

  21. Perform slot validity checks in a separate pass over expression.

  22. Rely on executor utils to build targetlist for DML RETURNING.

  23. Refer to OS X as "macOS", except for the port name which is still "darwin".