Horton

Re: JIT compiling with LLVM v9.0

David Fetter <david@fetter.org>

View thread View on postgr.es
From: David Fetter <david@fetter.org>
To: Andres Freund <andres@anarazel.de>
Cc: Konstantin Knizhnik <k.knizhnik@postgrespro.ru>, pgsql-hackers@lists.postgresql.org
Date: 2018-01-26T14:52:22Z
Lists: pgsql-hackers 
On Thu, Jan 25, 2018 at 11:20:28AM -0800, Andres Freund wrote:
> On 2018-01-25 18:40:53 +0300, Konstantin Knizhnik wrote:
> > Another question is whether it is sensible to redundantly do
> > expensive work (llvm compilation) in all backends.
> 
> Right now we kinda have to, but I really want to get rid of that.
> There's some pointers included as constants in the generated code. I
> plan to work on getting rid of that requirement, but after getting
> the basics in (i.e. realistically not this release).  Even after
> that I'm personally much more interested in caching the generated
> code inside a backend, rather than across backends.   Function
> addresses et al being different between backends would add some
> complications, can be overcome, but I'm doubtful it's immediately
> worth it.

If we go with threading for this part, sharing that state may be
simpler.  It seems a lot of work is going into things that threading
does at a much lower developer cost, but that's a different
conversation.

> > So before starting code generation, ExecReadyCompiledExpr can first
> > build signature and check if correspondent library is already present.
> > Also it will be easier to control space used by compiled libraries in
> > this
> 
> Right, I definitely think we want to do that at some point not too far
> away in the future. That makes the applicability of JITing much broader.
> 
> More advanced forms of this are that you JIT in the background for
> frequently executed code (so not to incur latency the first time
> somebody executes). Aand/or that you emit unoptimized code the first
> time through, which is quite quick, and run the optimizer after the
> query has been executed a number of times.

Both sound pretty neat.

Best,
David.
-- 
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

Commits

  1. Improve JIT docs.

  2. Add documentation for the JIT feature.

  3. Add EXPLAIN support for JIT.

  4. Add inlining support to LLVM JIT provider.

  5. JIT tuple deforming in LLVM JIT provider.

  6. Add FIELDNO_* macro designating offset into structs required for JIT.

  7. Add expression compilation support to LLVM JIT provider.

  8. Expand list of synchronized types and functions in LLVM JIT provider.

  9. Add helpers for emitting LLVM IR.

  10. Basic planner and executor integration for JIT.

  11. Debugging and profiling support for LLVM JIT provider.

  12. Support for optimizing and emitting code in LLVM JIT provider.

  13. Add file containing extensions of the LLVM C API.

  14. Basic JIT provider and error handling infrastructure.

  15. Add configure infrastructure (--with-llvm) to enable LLVM support.

  16. Add C++ support to configure.

  17. Add PGAC_PROG_VARCC_VARFLAGS_OPT autoconf macro.

  18. Fix VM buffer pin management in heap_lock_updated_tuple_rec().

  19. Allow tupleslots to have a fixed tupledesc, use in executor nodes.

  20. Expression evaluation based aggregate transition invocation.

  21. Perform slot validity checks in a separate pass over expression.

  22. Rely on executor utils to build targetlist for DML RETURNING.

  23. Refer to OS X as "macOS", except for the port name which is still "darwin".