Horton

Re: JIT compiling with LLVM v11

Catalin Iacob <iacobcatalin@gmail.com>

View thread View on postgr.es
From: Catalin Iacob <iacobcatalin@gmail.com>
To: Andres Freund <andres@anarazel.de>
Cc: Thomas Munro <thomas.munro@enterprisedb.com>, Pg Hackers <pgsql-hackers@postgresql.org>, Pierre Ducroquet <p.psql@pinaraf.info>
Date: 2018-03-15T16:19:23Z
Lists: pgsql-hackers 
On Thu, Mar 15, 2018 at 1:20 AM, Andres Freund <andres@anarazel.de> wrote:
> I don't really live in the RHEL world, but I wonder if
> https://developers.redhat.com/blog/2017/10/04/red-hat-adds-go-clangllvm-rust-compiler-toolsets-updates-gcc/
> is relevant?

Indeed. It might be a bit awkward for packagers to depend on something
from Software Collections, for example because they come as separate
trees in /opt that are by default not in your path or dynamic loader
path - one needs to run everything via a scl wrapper or source the
/opt/rh/llvm-toolset-7/enable file to get the appropriate PATH and
LD_LIBRARY_PATH settings, But it seems doable.

I just installed llvm-toolset-7 (the LLVM version is 4.0.1) on RHEL
7.4 and did a build of your tree at
475b4da439ae397345ab3df509e0e8eb26a8ff39. make installcheck passes for
both the default config and a server forced to jit everything (I
think) via:
jit_above_cost = '0'
jit_inline_above_cost = '0'
jit_optimize_above_cost = '0'

As a side note, this increases the runtime from approx 4 min to 18
min. Disabling jit completely with -1 in all of the above yields 3 min
48s, close to the default question raising maybe the question of how
much coverage does jit get with the default config.

The build was with the newer gcc 7.2.1 from the aforementioned
collections, I'll try the system gcc as well. I run a buildfarm animal
(katydid) on this RHEL. When JIT gets committed I'll make it use
--with-llvm against this Software Collections LLVM.

> Appears to be available on centos too
> https://www.softwarecollections.org/en/scls/rhscl/devtoolset-7/

Indeed they are available for CentOS as well.

Commits

  1. Improve JIT docs.

  2. Add documentation for the JIT feature.

  3. Add EXPLAIN support for JIT.

  4. Add inlining support to LLVM JIT provider.

  5. JIT tuple deforming in LLVM JIT provider.

  6. Add FIELDNO_* macro designating offset into structs required for JIT.

  7. Add expression compilation support to LLVM JIT provider.

  8. Expand list of synchronized types and functions in LLVM JIT provider.

  9. Add helpers for emitting LLVM IR.

  10. Basic planner and executor integration for JIT.

  11. Debugging and profiling support for LLVM JIT provider.

  12. Support for optimizing and emitting code in LLVM JIT provider.

  13. Add file containing extensions of the LLVM C API.

  14. Basic JIT provider and error handling infrastructure.

  15. Add configure infrastructure (--with-llvm) to enable LLVM support.

  16. Add C++ support to configure.

  17. Add PGAC_PROG_VARCC_VARFLAGS_OPT autoconf macro.

  18. Fix VM buffer pin management in heap_lock_updated_tuple_rec().

  19. Allow tupleslots to have a fixed tupledesc, use in executor nodes.

  20. Expression evaluation based aggregate transition invocation.

  21. Perform slot validity checks in a separate pass over expression.

  22. Rely on executor utils to build targetlist for DML RETURNING.

  23. Refer to OS X as "macOS", except for the port name which is still "darwin".