Horton

Re: JIT compiling with LLVM v11

Andres Freund <andres@anarazel.de>

View thread View on postgr.es
From: Andres Freund <andres@anarazel.de>
To: Catalin Iacob <iacobcatalin@gmail.com>
Cc: Thomas Munro <thomas.munro@enterprisedb.com>, Pg Hackers <pgsql-hackers@postgresql.org>, Pierre Ducroquet <p.psql@pinaraf.info>
Date: 2018-03-15T16:25:34Z
Lists: pgsql-hackers 
Hi,

On 2018-03-15 17:19:23 +0100, Catalin Iacob wrote:
> On Thu, Mar 15, 2018 at 1:20 AM, Andres Freund <andres@anarazel.de> wrote:
> > I don't really live in the RHEL world, but I wonder if
> > https://developers.redhat.com/blog/2017/10/04/red-hat-adds-go-clangllvm-rust-compiler-toolsets-updates-gcc/
> > is relevant?
> 
> Indeed. It might be a bit awkward for packagers to depend on something
> from Software Collections, for example because they come as separate
> trees in /opt that are by default not in your path or dynamic loader
> path - one needs to run everything via a scl wrapper or source the
> /opt/rh/llvm-toolset-7/enable file to get the appropriate PATH and
> LD_LIBRARY_PATH settings, But it seems doable.

It'd be just for clang, and they're not *forced* to do it, it's an
optional dependency. So I think I'm ok with that.


> I just installed llvm-toolset-7 (the LLVM version is 4.0.1) on RHEL
> 7.4 and did a build of your tree at
> 475b4da439ae397345ab3df509e0e8eb26a8ff39. make installcheck passes for
> both the default config and a server forced to jit everything (I
> think) via:
> jit_above_cost = '0'
> jit_inline_above_cost = '0'
> jit_optimize_above_cost = '0'
> 
> As a side note, this increases the runtime from approx 4 min to 18
> min.

Sure, that jits everything, which is obviously pointless to do for
performancereasons. Especially SQL functions play very badly, because
they're replanned every execution.  But it's good for testing ;)


> Disabling jit completely with -1 in all of the above yields 3 min
> 48s, close to the default question raising maybe the question of how
> much coverage does jit get with the default config.

A bit, but not hugely so. I'm not too concerned about that. I plan to
stand up a few buildfarm animals testing JITing with everything on w/
various LLVM versions.


> The build was with the newer gcc 7.2.1 from the aforementioned
> collections, I'll try the system gcc as well. I run a buildfarm animal
> (katydid) on this RHEL. When JIT gets committed I'll make it use
> --with-llvm against this Software Collections LLVM.

Cool! Thanks for testing!

Greetings,

Andres Freund

Commits

  1. Improve JIT docs.

  2. Add documentation for the JIT feature.

  3. Add EXPLAIN support for JIT.

  4. Add inlining support to LLVM JIT provider.

  5. JIT tuple deforming in LLVM JIT provider.

  6. Add FIELDNO_* macro designating offset into structs required for JIT.

  7. Add expression compilation support to LLVM JIT provider.

  8. Expand list of synchronized types and functions in LLVM JIT provider.

  9. Add helpers for emitting LLVM IR.

  10. Basic planner and executor integration for JIT.

  11. Debugging and profiling support for LLVM JIT provider.

  12. Support for optimizing and emitting code in LLVM JIT provider.

  13. Add file containing extensions of the LLVM C API.

  14. Basic JIT provider and error handling infrastructure.

  15. Add configure infrastructure (--with-llvm) to enable LLVM support.

  16. Add C++ support to configure.

  17. Add PGAC_PROG_VARCC_VARFLAGS_OPT autoconf macro.

  18. Fix VM buffer pin management in heap_lock_updated_tuple_rec().

  19. Allow tupleslots to have a fixed tupledesc, use in executor nodes.

  20. Expression evaluation based aggregate transition invocation.

  21. Perform slot validity checks in a separate pass over expression.

  22. Rely on executor utils to build targetlist for DML RETURNING.

  23. Refer to OS X as "macOS", except for the port name which is still "darwin".