WIP: Faster Expression Processing v4
Andres Freund <andres@anarazel.de>
From: Andres Freund <andres@anarazel.de>
To: pgsql-hackers@postgresql.org
Cc: Heikki Linnakangas <hlinnaka@iki.fi>, Doug Doole <ddoole@salesforce.com>
Date: 2017-02-09T00:33:12Z
Lists: pgsql-hackers
Attachments
- 0001-Add-expression-dependencies-on-composite-type-whole-.patch.gz (application/x-patch-gzip) patch 0001
- 0002-Make-get_last_attnums-more-generic.patch.gz (application/x-patch-gzip) patch 0002
- 0003-Add-autoconf-test-for-computed-goto-support.patch.gz (application/x-patch-gzip) patch 0003
- 0004-WIP-Faster-expression-processing-and-targetlist-proj.patch.gz (application/x-patch-gzip) patch 0004
- 0005-WIP-Add-configure-infrastructure-to-enable-LLVM.patch.gz (application/x-patch-gzip) patch 0005
- 0006-WIP-Beginning-of-a-LLVM-JIT-infrastructure.patch.gz (application/x-patch-gzip) patch 0006
- 0007-WIP-JITed-expression-evaluation.patch.gz (application/x-patch-gzip) patch 0007
Hi, Here's an updated version of this patchset (now that SRFs are implemented "properly"). This is a large patchset, with later patches being more experimental than earlier ones. 0001: Add expression dependencies on composite type / whole row components. This patch later allows to remove some execution time check (in ExecEvalScalarVar / ExecEvalFieldSelect), which'd be inconvenient to do when JITed. 0002: Make get_last_attnums more generic. What it says. 0003: autoconf test for computed goto support. The faster expression evaluation implementation uses computed gotos when available. So we need to check whether the current compiler supports those. 0004: WIP: Faster expression processing and targetlist projection. The most important patch of the series. Changes expression evaluation from being tree-walk based as done in execQual.c, into being a series of instructions. Then adds an opcode dispatch based interpreter for that mini language. That allows to later add different implementations. Projection is now a special case for expression evaluation. 0005: WIP: Add configure infrastructure to enable LLVM. 0006: WIP: Beginning of a LLVM JIT infrastructure. Needs work. 0007: WIP: JITed expression evaluation. This finally performs runtime compilation of expressions. I think up to 0004 things are getting pretty close to being ready. There's some minor FIXMEs that need to addressed (primarily around CaseTestExpr/CoerceToDomainValue when invoked outside of plans), and some more comment work. The biggest remaining piece of work in the whole series is 0006. Right compiled expressions can be leaked, which is obviously not ok. So I need to write ResourceOwner integration. 0007 covers nearly all types of expressions /queries (function usage isn't supported right now, and there's two missing calls to MakeExpandedObjectReadOnly). There's a lot more that could be done for efficiency (generating smarter LLVM IR basically), but we already see quite significant performance wins. I have later patches that - centralizes tuple deforming logic into slot_deform_logic from slot_getattr, slot_getsomeatts,slot_getallattrs and such - add JITing of tuple deforming - add JITing of aggregation transitions this works by converting transition functions into expression opcodes, and then adding JIT support for those new instructions (EEO_AGG_FILTER / STRICT_INPUT_CHECK / INIT_TRANS / STRICT_TRANS_CHECK / PLAIN_TRANS / ORDERED_TRANS_DATUM / ORDERED_TRANS_TUPLE) but the above 7 patches already are large enough imo. For now 0004 is I think the patch that really could use a look by somebody but me - it's quite a radical departure from how things currently look in master, and it's a fairly large patch. I don't really see a good way to break it down into useful parts :( (the projection changes are the only thing that I can see usefully being done separately). Regards, Andres
Commits
-
Improve performance of ExecEvalWholeRowVar.
- 2f0903ea1965 10.0 landed
-
Remove unreachable code in expression evaluation.
- ad46a2aa79f9 10.0 landed
-
Faster expression evaluation and targetlist projection.
- b8d7f053c5c2 10.0 landed
-
Avoid syntax error on platforms that have neither LOCALE_T nor ICU.
- 457a44487328 10.0 cited
-
Add configure test to see if the C compiler has gcc-style computed gotos.
- bc18126a6bcb 10.0 landed
-
Improve regression test coverage for TID scanning.
- be6c3d19fd45 10.0 landed
-
Improve expression evaluation test coverage.
- ce38949ba23a 10.0 landed
-
Fix two errors with nested CASE/WHEN constructs.
- f0c7b789ab12 9.6.0 cited