Thread
Commits
-
jit: Don't inline functions that access thread-locals.
- b1c1b7af573b 14.0 landed
- 1cfc9f2ef61d 13.4 landed
- b2f759e35f75 12.8 landed
- e515943acc06 11.13 landed
- 0207d5fbebed 15.0 landed
-
Re: BUG #16696: Backend crash in llvmjit
Dmitry Marakasov <amdmi3@amdmi3.ru> — 2020-11-02T21:45:09Z
* PG Bug reporting form (noreply@postgresql.org) wrote: > Environment details: > - FreeBSD 12.1 amd64 > - PostgreSQL 13.0 (built from FreeBSD ports) > - llvm-10.0.1 (build from FreeBSD ports) My bad, it's actually llvm-9.0.1. Multiple llvm versions are installed on the system, and PostgreSQL uses llvm9: ldd /usr/local/lib/postgresql/llvmjit.so | grep LLVM libLLVM-9.so => /usr/local/llvm90/lib/libLLVM-9.so (0x800e00000) -- Dmitry Marakasov . 55B5 0596 FF1E 8D84 5F56 9510 D35A 80DD F9D2 F77D amdmi3@amdmi3.ru ..: https://github.com/AMDmi3
-
BUG #16696: Backend crash in llvmjit
The Post Office <noreply@postgresql.org> — 2020-11-02T21:47:17Z
The following bug has been logged on the website: Bug reference: 16696 Logged by: Dmitry Marakasov Email address: amdmi3@amdmi3.ru PostgreSQL version: 13.0 Operating system: FreeBSD 12.1 Description: Hi! I've ran into llvmjit related backend crash with a query involving unnesting a column which contains array of arrays in jsonb format. I've been able to isolate a query which triggers the problem: BEGIN; CREATE TEMP TABLE test AS SELECT ( SELECT jsonb_agg('["foofoofoo","foofoofoo","*","*","*","*","*","*","*","*",true,false]'::jsonb) FROM generate_series(1,1000) ) AS data FROM generate_series(1, 10000); SELECT COUNT(*) FROM( SELECT jsonb_array_elements(data)->>0, jsonb_array_elements(data)->>1, jsonb_array_elements(data)->>2, jsonb_array_elements(data)->>3, jsonb_array_elements(data)->>4, jsonb_array_elements(data)->>5, jsonb_array_elements(data)->>6, jsonb_array_elements(data)->>7, jsonb_array_elements(data)->>8, jsonb_array_elements(data)->>9, (jsonb_array_elements(data)->>10)::boolean, (jsonb_array_elements(data)->>11)::boolean FROM test ) AS t; ROLLBACK; The result is 100% reproduciby on my setup: % psql < test.sql BEGIN SELECT 10000 server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request. connection to server was lost Syslog: "pid 53852 (postgres), jid 0, uid 770: exited on signal 11 (core dumped)" Expected result which is achievable on PostgreSQL compiled without LLVM support: % psql < test.sql BEGIN SELECT 10000 count ---------- 10000000 (1 row) ROLLBACK Environment details: - FreeBSD 12.1 amd64 - PostgreSQL 13.0 (built from FreeBSD ports) - llvm-10.0.1 (build from FreeBSD ports) - version(): PostgreSQL 13.0 on amd64-portbld-freebsd12.1, compiled by FreeBSD clang version 8.0.1 (tags/RELEASE_801/final 366581) (based on LLVM 8.0.1), 64-bit - the config is as far as I can tell default: http://dpaste.com//AJ4RCJ7DG Note that the problem may be related to some buffer overflow, as using less rows in test table or less nested arrays in the jsonb field makes the problem go away. Backtrace: (lldb) bt * thread #1, name = 'postgres', stop reason = signal SIGSEGV * frame #0: 0x0000000810f4a70b frame #1: 0x000000080bfded8d llvmjit.so`ExecRunCompiledExpr(state=0x000000080b18a850, econtext=0x0000000801bc2a80, isNull=0x00007fffffffddd7) at llvmjit_expr.c:2424:9 frame #2: 0x00000000007dd70b postgres`ExecEvalExprSwitchContext(state=0x000000080b18a850, econtext=0x0000000801bc2a80, isNull=0x00007fffffffddd7) at executor.h:313:13 frame #3: 0x00000000007dd669 postgres`ExecProject(projInfo=0x000000080b18a848) at executor.h:347:9 frame #4: 0x00000000007dd42f postgres`ExecResult(pstate=0x0000000801bc2970) at nodeResult.c:136:10 frame #5: 0x00000000007a1035 postgres`ExecProcNodeFirst(node=0x0000000801bc2970) at execProcnode.c:450:9 frame #6: 0x00000000007b5e22 postgres`ExecProcNode(node=0x0000000801bc2970) at executor.h:245:9 frame #7: 0x00000000007b5a4c postgres`fetch_input_tuple(aggstate=0x0000000801bc2398) at nodeAgg.c:589:10 frame #8: 0x00000000007b5678 postgres`agg_retrieve_direct(aggstate=0x0000000801bc2398) at nodeAgg.c:2356:17 frame #9: 0x00000000007b250e postgres`ExecAgg(pstate=0x0000000801bc2398) at nodeAgg.c:2171:14 frame #10: 0x00000000007a1035 postgres`ExecProcNodeFirst(node=0x0000000801bc2398) at execProcnode.c:450:9 frame #11: 0x0000000000799db2 postgres`ExecProcNode(node=0x0000000801bc2398) at executor.h:245:9 frame #12: 0x0000000000796161 postgres`ExecutePlan(estate=0x0000000801bc2118, planstate=0x0000000801bc2398, use_parallel_mode=false, operation=CMD_SELECT, sendTuples=true, numberTuples=0, direction=ForwardScanDirection, dest=0x000000080b565f00, execute_once=true) at execMain.c:1646:10 frame #13: 0x000000000079602e postgres`standard_ExecutorRun(queryDesc=0x000000080220e918, direction=ForwardScanDirection, count=0, execute_once=true) at execMain.c:364:3 frame #14: 0x0000000000795e84 postgres`ExecutorRun(queryDesc=0x000000080220e918, direction=ForwardScanDirection, count=0, execute_once=true) at execMain.c:308:3 frame #15: 0x00000000009f4928 postgres`PortalRunSelect(portal=0x000000080b0c3118, forward=true, count=0, dest=0x000000080b565f00) at pquery.c:912:4 frame #16: 0x00000000009f438d postgres`PortalRun(portal=0x000000080b0c3118, count=9223372036854775807, isTopLevel=true, run_once=true, dest=0x000000080b565f00, altdest=0x000000080b565f00, qc=0x00007fffffffe3a0) at pquery.c:756:18 frame #17: 0x00000000009efc10 postgres`exec_simple_query(query_string="SELECT COUNT(*) FROM(\n\t\tSELECT\n\t\t\tjsonb_array_elements(data)->>0,\n\t\t\tjsonb_array_elements(data)->>1,\n\t\t\tjsonb_array_elements(data)->>2,\n\t\t\tjsonb_array_elements(data)->>3,\n\t\t\tjsonb_array_elements(data)->>4,\n\t\t\tjsonb_array_elements(data)->>5,\n\t\t\tjsonb_array_elements(data)->>6,\n\t\t\tjsonb_array_elements(data)->>7,\n\t\t\tjsonb_array_elements(data)->>8,\n\t\t\tjsonb_array_elements(data)->>9,\n\t\t\t(jsonb_array_elements(data)->>10)::boolean,\n\t\t\t(jsonb_array_elements(data)->>11)::boolean\n\t\tFROM test\n\t) AS t;") at postgres.c:1239:10 frame #18: 0x00000000009eeec9 postgres`PostgresMain(argc=1, argv=0x000000080b0ac730, dbname="repology", username="repology") at postgres.c:4315:7 frame #19: 0x000000000092bcb7 postgres`BackendRun(port=0x000000080b0a5000) at postmaster.c:4536:2 frame #20: 0x000000000092b090 postgres`BackendStartup(port=0x000000080b0a5000) at postmaster.c:4220:3 frame #21: 0x000000000092a073 postgres`ServerLoop at postmaster.c:1739:7 frame #22: 0x0000000000927d52 postgres`PostmasterMain(argc=3, argv=0x00007fffffffeb48) at postmaster.c:1412:11 frame #23: 0x0000000000819bcf postgres`main(argc=3, argv=0x00007fffffffeb48) at main.c:210:3 frame #24: 0x00000000004cb10f postgres`_start(ap=<unavailable>, cleanup=<unavailable>) at crt1.c:76:7 Some locals at #1 (lldb) print *state (ExprState) $6 = { tag = T_ExprState flags = '\0' resnull = false resvalue = 34388814424 resultslot = 0x0000000801bc3ec0 steps = 0x000000080b4a7f38 evalfunc = 0x0000000810f4a120 expr = 0x000000080b564940 evalfunc_private = 0x000000080b4a9fe0 steps_len = 39 steps_alloc = 64 parent = 0x0000000801bc2970 ext_params = 0x0000000000000000 innermost_caseval = 0x0000000000000000 innermost_casenull = 0x0000000000000000 innermost_domainval = 0x0000000000000000 innermost_domainnull = 0x0000000000000000 } (lldb) print *econtext (ExprContext) $7 = { type = T_ExprContext ecxt_scantuple = 0x0000000000000000 ecxt_innertuple = 0x0000000000000000 ecxt_outertuple = 0x0000000801bc31e0 ecxt_per_query_memory = 0x0000000801bc2000 ecxt_per_tuple_memory = 0x0000000801bba000 ecxt_param_exec_vals = 0x0000000000000000 ecxt_param_list_info = 0x0000000000000000 ecxt_aggvalues = 0x0000000000000000 ecxt_aggnulls = 0x0000000000000000 caseValue_datum = 0 caseValue_isNull = true domainValue_datum = 0 domainValue_isNull = true ecxt_estate = 0x0000000801bc2118 ecxt_callbacks = 0x0000000000000000 } (lldb) print *isNull (bool) $8 = false (lldb) print *(CompiledExprState*)state->evalfunc_private (CompiledExprState) $16 = { context = 0x000000080b126050 funcname = 0x000000080b4a90d8 "evalexpr_0_2" } -
Re: BUG #16696: Backend crash in llvmjit
Andres Freund <andres@anarazel.de> — 2020-11-03T00:00:37Z
Hi On Mon, Nov 2, 2020, at 13:45, Dmitry Marakasov wrote: > * PG Bug reporting form (noreply@postgresql.org) wrote: > > > Environment details: > > - FreeBSD 12.1 amd64 > > - PostgreSQL 13.0 (built from FreeBSD ports) > > - llvm-10.0.1 (build from FreeBSD ports) > > My bad, it's actually llvm-9.0.1. Multiple llvm versions are installed on > the system, and PostgreSQL uses llvm9: > > ldd /usr/local/lib/postgresql/llvmjit.so | grep LLVM > libLLVM-9.so => /usr/local/llvm90/lib/libLLVM-9.so (0x800e00000) Could you try generating a backtrace after turning jit_debugging_support on? That might give a bit more information. I'll check once I'm home whether I can reproduce in my environment. Regards, Andres
-
Re: BUG #16696: Backend crash in llvmjit
Dmitry Marakasov <amdmi3@amdmi3.ru> — 2020-11-03T00:38:35Z
* Andres Freund (andres@anarazel.de) wrote: > On Mon, Nov 2, 2020, at 13:45, Dmitry Marakasov wrote: > > * PG Bug reporting form (noreply@postgresql.org) wrote: > > > > > Environment details: > > > - FreeBSD 12.1 amd64 > > > - PostgreSQL 13.0 (built from FreeBSD ports) > > > - llvm-10.0.1 (build from FreeBSD ports) > > > > My bad, it's actually llvm-9.0.1. Multiple llvm versions are installed on > > the system, and PostgreSQL uses llvm9: > > > > ldd /usr/local/lib/postgresql/llvmjit.so | grep LLVM > > libLLVM-9.so => /usr/local/llvm90/lib/libLLVM-9.so (0x800e00000) > > Could you try generating a backtrace after turning jit_debugging_support on? That might give a bit more information. It doesn't give too much apart from the function name. Here's gdb output: #0 0x0000000810f5c70b in evalexpr_0_2 () No symbol table info available. #1 0x000000080bfded8d in ExecRunCompiledExpr (state=0x80b18a850, econtext=0x801bc2a80, isNull=0x7fffffffddd7) at llvmjit_expr.c:2424 cstate = 0x80b4a9fe0 func = 0x810f5c120 <evalexpr_0_2> #2 0x00000000007dd70b in ExecEvalExprSwitchContext (state=0x80b18a850, econtext=0x801bc2a80, isNull=0x7fffffffddd7) at ../../../src/include/executor/executor.h:313 retDatum = 140737488346496 oldContext = 0x801bc2000 #3 0x00000000007dd669 in ExecProject (projInfo=0x80b18a848) at ../../../src/include/executor/executor.h:347 econtext = 0x801bc2a80 state = 0x80b18a850 slot = 0x801bc3ec0 isnull = false #4 0x00000000007dd42f in ExecResult (pstate=0x801bc2970) at nodeResult.c:136 node = 0x801bc2970 outerTupleSlot = 0x801bc31e0 outerPlan = 0x801bc2b10 econtext = 0x801bc2a80 #5 0x00000000007a1035 in ExecProcNodeFirst (node=0x801bc2970) at execProcnode.c:450 No locals. #6 0x00000000007b5e22 in ExecProcNode (node=0x801bc2970) at ../../../src/include/executor/executor.h:245 No locals. #7 0x00000000007b5a4c in fetch_input_tuple (aggstate=0x801bc2398) at nodeAgg.c:589 slot = 0x7a9ef7 <ReScanExprContext+39> #8 0x00000000007b5678 in agg_retrieve_direct (aggstate=0x801bc2398) at nodeAgg.c:2356 node = 0x801bb5dd0 econtext = 0x801bc28e0 tmpcontext = 0x801bc27c0 peragg = 0x80b4aa6f0 pergroups = 0x80b4aa990 outerslot = 0x1a firstSlot = 0x80b4aa000 result = 0xffff0b4aa12c hasGroupingSets = false numGroupingSets = 1 currentSet = 32767 nextSetSize = 0 numReset = 1 i = 1 #9 0x00000000007b250e in ExecAgg (pstate=0x801bc2398) at nodeAgg.c:2171 node = 0x801bc2398 result = 0x0 #10 0x00000000007a1035 in ExecProcNodeFirst (node=0x801bc2398) at execProcnode.c:450 No locals. #11 0x0000000000799db2 in ExecProcNode (node=0x801bc2398) at ../../../src/include/executor/executor.h:245 No locals. #12 0x0000000000796161 in ExecutePlan (estate=0x801bc2118, planstate=0x801bc2398, use_parallel_mode=false, operation=CMD_SELECT, sendTuples=true, numberTuples=0, direction=ForwardScanDirection, dest=0x80b565f00, execute_once=true) at execMain.c:1646 slot = 0x80b565f50 current_tuple_count = 0 #13 0x000000000079602e in standard_ExecutorRun (queryDesc=0x80220e918, direction=ForwardScanDirection, count=0, execute_once=true) at execMain.c:364 estate = 0x801bc2118 operation = CMD_SELECT dest = 0x80b565f00 sendTuples = true oldcontext = 0x80220e800 #14 0x0000000000795e84 in ExecutorRun (queryDesc=0x80220e918, direction=ForwardScanDirection, count=0, execute_once=true) at execMain.c:308 No locals. #15 0x00000000009f4928 in PortalRunSelect (portal=0x80b0c3118, forward=true, count=0, dest=0x80b565f00) at pquery.c:912 queryDesc = 0x80220e918 direction = ForwardScanDirection nprocessed = 294333 #16 0x00000000009f438d in PortalRun (portal=0x80b0c3118, count=9223372036854775807, isTopLevel=true, run_once=true, dest=0x80b565f00, altdest=0x80b565f00, qc=0x7fffffffe3a0) at pquery.c:756 _save_exception_stack = 0x7fffffffe580 _save_context_stack = 0x0 _local_sigjmp_buf = {{_sjb = {10437281, -4571079317099439115, 140737488347448, 140737488347776, 140737488350016, 0, 140737488350024, 140737488350056, -64641, 2, 4294967297, 34359738368}}} _do_rethrow = false result = false nprocessed = 34395449568 saveTopTransactionResourceOwner = 0x80b0ae3a0 saveTopTransactionContext = 0x80b0cb000 saveActivePortal = 0x0 saveResourceOwner = 0x80b0ae3a0 savePortalContext = 0x0 saveMemoryContext = 0x80b0cb000 #17 0x00000000009efc10 in exec_simple_query ( query_string=0x801b87118 "SELECT COUNT(*) FROM(\n\t\tSELECT\n\t\t\tjsonb_array_elements(data)->>0,\n\t\t\tjsonb_array_elements(data)->>1,\n\t\t\tjsonb_array_elements(data)->>2,\n\t\t\tjsonb_array_elements(data)->>3,\n\t\t\tjsonb_array_elements(data)"...) at postgres.c:1239 snapshot_set = true per_parsetree_context = 0x0 plantree_list = 0x80b565eb0 parsetree = 0x801b8ecd0 commandTag = CMDTAG_SELECT qc = {commandTag = CMDTAG_UNKNOWN, nprocessed = 0} querytree_list = 0x801bb9160 portal = 0x80b0c3118 receiver = 0x80b565f00 format = 0 parsetree_item__state = {l = 0x801b8ed00, i = 0} dest = DestRemote oldcontext = 0x80b0cb000 parsetree_list = 0x801b8ed00 parsetree_item = 0x801b8ed18 save_log_statement_stats = false was_logged = false use_implicit_block = false msec_str = "\340\343\377\377\377\177\000\000\022S\201\000\000\000\000\000\377q\t\000\000\000\000\000\365\033'h\356\001\000" #18 0x00000000009eeec9 in PostgresMain (argc=1, argv=0x80b0ac730, dbname=0x80b0ac6a8 "repology", username=0x80b0ac688 "repology") at postgres.c:4315 query_string = 0x801b87118 "SELECT COUNT(*) FROM(\n\t\tSELECT\n\t\t\tjsonb_array_elements(data)->>0,\n\t\t\tjsonb_array_elements(data)->>1,\n\t\t\tjsonb_array_elements(data)->>2,\n\t\t\tjsonb_array_elements(data)->>3,\n\t\t\tjsonb_array_elements(data)"... firstchar = 81 input_message = { data = 0x801b87118 "SELECT COUNT(*) FROM(\n\t\tSELECT\n\t\t\tjsonb_array_elements(data)->>0,\n\t\t\tjsonb_array_elements(data)->>1,\n\t\t\tjsonb_array_elements(data)->>2,\n\t\t\tjsonb_array_elements(data)->>3,\n\t\t\tjsonb_array_elements(data)"..., len = 495, maxlen = 1024, cursor = 495} local_sigjmp_buf = {{_sjb = {10414768, 3, 140737488348136, 140737488348704, 140737488350016, 0, 140737488350024, 140737488350056, 34388575103, 0, 0, 1}}} send_ready_for_query = false disable_idle_in_transaction_timeout = false #19 0x000000000092bcb7 in BackendRun (port=0x80b0a5000) at postmaster.c:4536 av = 0x80b0ac730 maxac = 2 ac = 1 i = 1 #20 0x000000000092b090 in BackendStartup (port=0x80b0a5000) at postmaster.c:4220 bn = 0x801b97510 pid = 0 #21 0x000000000092a073 in ServerLoop () at postmaster.c:1739 port = 0x80b0a5000 i = 2 rmask = {__fds_bits = {256, 0 <repeats 15 times>}} selres = 1 now = 1604363041 readmask = {__fds_bits = {448, 0 <repeats 15 times>}} nSockets = 9 last_lockfile_recheck_time = 1604363041 last_touch_time = 1604363041 #22 0x0000000000927d52 in PostmasterMain (argc=3, argv=0x7fffffffeb48) at postmaster.c:1412 opt = -1 status = 0 userDoption = 0x800c881a0 "/var/db/postgres/data13" listen_addr_saved = true i = 64 output_config_variable = 0x0 #23 0x0000000000819bcf in main (argc=3, argv=0x7fffffffeb48) at main.c:210 do_check_root = true Disassembly: Dump of assembler code for function evalexpr_0_2: 0x0000000810f5c120 <+0>: push %rbp 0x0000000810f5c121 <+1>: mov %rsp,%rbp 0x0000000810f5c124 <+4>: push %r15 0x0000000810f5c126 <+6>: push %r14 0x0000000810f5c128 <+8>: push %r13 0x0000000810f5c12a <+10>: push %r12 0x0000000810f5c12c <+12>: push %rbx 0x0000000810f5c12d <+13>: sub $0x48,%rsp 0x0000000810f5c131 <+17>: mov %rdx,-0x68(%rbp) 0x0000000810f5c135 <+21>: mov %rdi,%r12 0x0000000810f5c138 <+24>: movabs $0x80b18a855,%r13 0x0000000810f5c142 <+34>: mov 0x18(%rsi),%rax 0x0000000810f5c146 <+38>: mov 0x10(%rdi),%rcx 0x0000000810f5c14a <+42>: mov 0x18(%rax),%r14 0x0000000810f5c14e <+46>: mov 0x20(%rax),%r15 0x0000000810f5c152 <+50>: mov 0x18(%rcx),%rbx 0x0000000810f5c156 <+54>: mov 0x20(%rcx),%rdx 0x0000000810f5c15a <+58>: mov (%r14),%rax 0x0000000810f5c15d <+61>: mov (%r15),%cl 0x0000000810f5c160 <+64>: mov %rax,0xf3(%r13) 0x0000000810f5c167 <+71>: mov %cl,0xfb(%r13) 0x0000000810f5c16e <+78>: movb $0x1,0x0(%r13) 0x0000000810f5c173 <+83>: cmp $0x1,%cl 0x0000000810f5c176 <+86>: mov %rdx,-0x30(%rbp) 0x0000000810f5c17a <+90>: je 0x810f5c1b4 <evalexpr_0_2+148> 0x0000000810f5c17c <+92>: cmpb $0x1,0x10b(%r13) 0x0000000810f5c184 <+100>: je 0x810f5c1b4 <evalexpr_0_2+148> 0x0000000810f5c186 <+102>: movb $0x0,0xef(%r13) 0x0000000810f5c18e <+110>: lea 0xd3(%r13),%rdi 0x0000000810f5c195 <+117>: movabs $0xaa0b40,%rax 0x0000000810f5c19f <+127>: callq *%rax 0x0000000810f5c1a1 <+129>: mov -0x30(%rbp),%rdx 0x0000000810f5c1a5 <+133>: mov 0xef(%r13),%cl 0x0000000810f5c1ac <+140>: mov %rax,0x3(%r13) 0x0000000810f5c1b0 <+144>: mov %cl,0x0(%r13) 0x0000000810f5c1b4 <+148>: mov 0x8(%r12),%rax 0x0000000810f5c1b9 <+153>: mov 0x5(%r12),%cl 0x0000000810f5c1be <+158>: mov %cl,(%rdx) 0x0000000810f5c1c0 <+160>: test %cl,%cl 0x0000000810f5c1c2 <+162>: jne 0x810f5c1da <evalexpr_0_2+186> 0x0000000810f5c1c4 <+164>: cmpb $0x1,(%rax) 0x0000000810f5c1c7 <+167>: jne 0x810f5c1d7 <evalexpr_0_2+183> 0x0000000810f5c1c9 <+169>: cmpb $0x3,0x1(%rax) 0x0000000810f5c1cd <+173>: jne 0x810f5c1d7 <evalexpr_0_2+183> 0x0000000810f5c1cf <+175>: mov 0x2(%rax),%rax 0x0000000810f5c1d3 <+179>: add $0x22,%rax 0x0000000810f5c1d7 <+183>: mov %rax,(%rbx) 0x0000000810f5c1da <+186>: mov (%r14),%rax 0x0000000810f5c1dd <+189>: mov (%r15),%cl 0x0000000810f5c1e0 <+192>: mov %rax,0x5a3(%r13) 0x0000000810f5c1e7 <+199>: mov %cl,0x5ab(%r13) 0x0000000810f5c1ee <+206>: movb $0x1,0x0(%r13) 0x0000000810f5c1f3 <+211>: cmp $0x1,%cl 0x0000000810f5c1f6 <+214>: je 0x810f5c230 <evalexpr_0_2+272> 0x0000000810f5c1f8 <+216>: cmpb $0x1,0x5bb(%r13) 0x0000000810f5c200 <+224>: je 0x810f5c230 <evalexpr_0_2+272> 0x0000000810f5c202 <+226>: movb $0x0,0x59f(%r13) 0x0000000810f5c20a <+234>: lea 0x583(%r13),%rdi 0x0000000810f5c211 <+241>: movabs $0xaa0b40,%rax 0x0000000810f5c21b <+251>: callq *%rax 0x0000000810f5c21d <+253>: mov -0x30(%rbp),%rdx 0x0000000810f5c221 <+257>: mov 0x59f(%r13),%cl 0x0000000810f5c228 <+264>: mov %rax,0x3(%r13) 0x0000000810f5c22c <+268>: mov %cl,0x0(%r13) 0x0000000810f5c230 <+272>: mov 0x8(%r12),%rax 0x0000000810f5c235 <+277>: mov 0x5(%r12),%cl 0x0000000810f5c23a <+282>: mov %cl,0x1(%rdx) 0x0000000810f5c23d <+285>: test %cl,%cl 0x0000000810f5c23f <+287>: jne 0x810f5c258 <evalexpr_0_2+312> 0x0000000810f5c241 <+289>: cmpb $0x1,(%rax) 0x0000000810f5c244 <+292>: jne 0x810f5c254 <evalexpr_0_2+308> 0x0000000810f5c246 <+294>: cmpb $0x3,0x1(%rax) 0x0000000810f5c24a <+298>: jne 0x810f5c254 <evalexpr_0_2+308> 0x0000000810f5c24c <+300>: mov 0x2(%rax),%rax 0x0000000810f5c250 <+304>: add $0x22,%rax 0x0000000810f5c254 <+308>: mov %rax,0x8(%rbx) 0x0000000810f5c258 <+312>: mov (%r14),%rax 0x0000000810f5c25b <+315>: mov (%r15),%cl 0x0000000810f5c25e <+318>: mov %rax,0x643(%r13) 0x0000000810f5c265 <+325>: mov %cl,0x64b(%r13) 0x0000000810f5c26c <+332>: movb $0x1,0x0(%r13) 0x0000000810f5c271 <+337>: cmp $0x1,%cl 0x0000000810f5c274 <+340>: je 0x810f5c2ae <evalexpr_0_2+398> 0x0000000810f5c276 <+342>: cmpb $0x1,0x65b(%r13) 0x0000000810f5c27e <+350>: je 0x810f5c2ae <evalexpr_0_2+398> 0x0000000810f5c280 <+352>: movb $0x0,0x63f(%r13) 0x0000000810f5c288 <+360>: lea 0x623(%r13),%rdi 0x0000000810f5c28f <+367>: movabs $0xaa0b40,%rax 0x0000000810f5c299 <+377>: callq *%rax 0x0000000810f5c29b <+379>: mov -0x30(%rbp),%rdx 0x0000000810f5c29f <+383>: mov 0x63f(%r13),%cl 0x0000000810f5c2a6 <+390>: mov %rax,0x3(%r13) 0x0000000810f5c2aa <+394>: mov %cl,0x0(%r13) 0x0000000810f5c2ae <+398>: mov 0x8(%r12),%rax 0x0000000810f5c2b3 <+403>: mov 0x5(%r12),%cl 0x0000000810f5c2b8 <+408>: mov %cl,0x2(%rdx) 0x0000000810f5c2bb <+411>: test %cl,%cl 0x0000000810f5c2bd <+413>: jne 0x810f5c2d6 <evalexpr_0_2+438> 0x0000000810f5c2bf <+415>: cmpb $0x1,(%rax) 0x0000000810f5c2c2 <+418>: jne 0x810f5c2d2 <evalexpr_0_2+434> 0x0000000810f5c2c4 <+420>: cmpb $0x3,0x1(%rax) 0x0000000810f5c2c8 <+424>: jne 0x810f5c2d2 <evalexpr_0_2+434> 0x0000000810f5c2ca <+426>: mov 0x2(%rax),%rax 0x0000000810f5c2ce <+430>: add $0x22,%rax 0x0000000810f5c2d2 <+434>: mov %rax,0x10(%rbx) 0x0000000810f5c2d6 <+438>: mov (%r14),%rax 0x0000000810f5c2d9 <+441>: mov (%r15),%cl 0x0000000810f5c2dc <+444>: mov %rax,0x6e3(%r13) 0x0000000810f5c2e3 <+451>: mov %cl,0x6eb(%r13) 0x0000000810f5c2ea <+458>: movb $0x1,0x0(%r13) 0x0000000810f5c2ef <+463>: cmp $0x1,%cl 0x0000000810f5c2f2 <+466>: je 0x810f5c32c <evalexpr_0_2+524> 0x0000000810f5c2f4 <+468>: cmpb $0x1,0x6fb(%r13) 0x0000000810f5c2fc <+476>: je 0x810f5c32c <evalexpr_0_2+524> 0x0000000810f5c2fe <+478>: movb $0x0,0x6df(%r13) 0x0000000810f5c306 <+486>: lea 0x6c3(%r13),%rdi 0x0000000810f5c30d <+493>: movabs $0xaa0b40,%rax 0x0000000810f5c317 <+503>: callq *%rax 0x0000000810f5c319 <+505>: mov -0x30(%rbp),%rdx 0x0000000810f5c31d <+509>: mov 0x6df(%r13),%cl 0x0000000810f5c324 <+516>: mov %rax,0x3(%r13) 0x0000000810f5c328 <+520>: mov %cl,0x0(%r13) 0x0000000810f5c32c <+524>: mov 0x8(%r12),%rax 0x0000000810f5c331 <+529>: mov 0x5(%r12),%cl 0x0000000810f5c336 <+534>: mov %cl,0x3(%rdx) 0x0000000810f5c339 <+537>: test %cl,%cl 0x0000000810f5c33b <+539>: jne 0x810f5c354 <evalexpr_0_2+564> 0x0000000810f5c33d <+541>: cmpb $0x1,(%rax) 0x0000000810f5c340 <+544>: jne 0x810f5c350 <evalexpr_0_2+560> 0x0000000810f5c342 <+546>: cmpb $0x3,0x1(%rax) 0x0000000810f5c346 <+550>: jne 0x810f5c350 <evalexpr_0_2+560> 0x0000000810f5c348 <+552>: mov 0x2(%rax),%rax 0x0000000810f5c34c <+556>: add $0x22,%rax 0x0000000810f5c350 <+560>: mov %rax,0x18(%rbx) 0x0000000810f5c354 <+564>: mov (%r14),%rax 0x0000000810f5c357 <+567>: mov (%r15),%cl 0x0000000810f5c35a <+570>: mov %rax,0x783(%r13) 0x0000000810f5c361 <+577>: mov %cl,0x78b(%r13) 0x0000000810f5c368 <+584>: movb $0x1,0x0(%r13) 0x0000000810f5c36d <+589>: cmp $0x1,%cl 0x0000000810f5c370 <+592>: je 0x810f5c3aa <evalexpr_0_2+650> 0x0000000810f5c372 <+594>: cmpb $0x1,0x79b(%r13) 0x0000000810f5c37a <+602>: je 0x810f5c3aa <evalexpr_0_2+650> 0x0000000810f5c37c <+604>: movb $0x0,0x77f(%r13) 0x0000000810f5c384 <+612>: lea 0x763(%r13),%rdi 0x0000000810f5c38b <+619>: movabs $0xaa0b40,%rax 0x0000000810f5c395 <+629>: callq *%rax 0x0000000810f5c397 <+631>: mov -0x30(%rbp),%rdx 0x0000000810f5c39b <+635>: mov 0x77f(%r13),%cl 0x0000000810f5c3a2 <+642>: mov %rax,0x3(%r13) 0x0000000810f5c3a6 <+646>: mov %cl,0x0(%r13) 0x0000000810f5c3aa <+650>: mov 0x8(%r12),%rax 0x0000000810f5c3af <+655>: mov 0x5(%r12),%cl 0x0000000810f5c3b4 <+660>: mov %cl,0x4(%rdx) 0x0000000810f5c3b7 <+663>: test %cl,%cl 0x0000000810f5c3b9 <+665>: jne 0x810f5c3d2 <evalexpr_0_2+690> 0x0000000810f5c3bb <+667>: cmpb $0x1,(%rax) 0x0000000810f5c3be <+670>: jne 0x810f5c3ce <evalexpr_0_2+686> 0x0000000810f5c3c0 <+672>: cmpb $0x3,0x1(%rax) 0x0000000810f5c3c4 <+676>: jne 0x810f5c3ce <evalexpr_0_2+686> 0x0000000810f5c3c6 <+678>: mov 0x2(%rax),%rax 0x0000000810f5c3ca <+682>: add $0x22,%rax 0x0000000810f5c3ce <+686>: mov %rax,0x20(%rbx) 0x0000000810f5c3d2 <+690>: mov (%r14),%rax 0x0000000810f5c3d5 <+693>: mov (%r15),%cl 0x0000000810f5c3d8 <+696>: mov %rax,0x823(%r13) 0x0000000810f5c3df <+703>: mov %cl,0x82b(%r13) 0x0000000810f5c3e6 <+710>: movb $0x1,0x0(%r13) 0x0000000810f5c3eb <+715>: cmp $0x1,%cl 0x0000000810f5c3ee <+718>: je 0x810f5c428 <evalexpr_0_2+776> 0x0000000810f5c3f0 <+720>: cmpb $0x1,0x83b(%r13) 0x0000000810f5c3f8 <+728>: je 0x810f5c428 <evalexpr_0_2+776> 0x0000000810f5c3fa <+730>: movb $0x0,0x81f(%r13) 0x0000000810f5c402 <+738>: lea 0x803(%r13),%rdi 0x0000000810f5c409 <+745>: movabs $0xaa0b40,%rax 0x0000000810f5c413 <+755>: callq *%rax 0x0000000810f5c415 <+757>: mov -0x30(%rbp),%rdx 0x0000000810f5c419 <+761>: mov 0x81f(%r13),%cl 0x0000000810f5c420 <+768>: mov %rax,0x3(%r13) 0x0000000810f5c424 <+772>: mov %cl,0x0(%r13) 0x0000000810f5c428 <+776>: mov 0x8(%r12),%rax 0x0000000810f5c42d <+781>: mov 0x5(%r12),%cl 0x0000000810f5c432 <+786>: mov %cl,0x5(%rdx) 0x0000000810f5c435 <+789>: test %cl,%cl 0x0000000810f5c437 <+791>: jne 0x810f5c450 <evalexpr_0_2+816> 0x0000000810f5c439 <+793>: cmpb $0x1,(%rax) 0x0000000810f5c43c <+796>: jne 0x810f5c44c <evalexpr_0_2+812> 0x0000000810f5c43e <+798>: cmpb $0x3,0x1(%rax) 0x0000000810f5c442 <+802>: jne 0x810f5c44c <evalexpr_0_2+812> 0x0000000810f5c444 <+804>: mov 0x2(%rax),%rax 0x0000000810f5c448 <+808>: add $0x22,%rax 0x0000000810f5c44c <+812>: mov %rax,0x28(%rbx) 0x0000000810f5c450 <+816>: mov (%r14),%rax 0x0000000810f5c453 <+819>: mov (%r15),%cl 0x0000000810f5c456 <+822>: mov %rax,0x10d3(%r13) 0x0000000810f5c45d <+829>: mov %cl,0x10db(%r13) 0x0000000810f5c464 <+836>: movb $0x1,0x0(%r13) 0x0000000810f5c469 <+841>: cmp $0x1,%cl 0x0000000810f5c46c <+844>: je 0x810f5c4a6 <evalexpr_0_2+902> 0x0000000810f5c46e <+846>: cmpb $0x1,0x10eb(%r13) 0x0000000810f5c476 <+854>: je 0x810f5c4a6 <evalexpr_0_2+902> 0x0000000810f5c478 <+856>: movb $0x0,0x10cf(%r13) 0x0000000810f5c480 <+864>: lea 0x10b3(%r13),%rdi 0x0000000810f5c487 <+871>: movabs $0xaa0b40,%rax 0x0000000810f5c491 <+881>: callq *%rax 0x0000000810f5c493 <+883>: mov -0x30(%rbp),%rdx 0x0000000810f5c497 <+887>: mov 0x10cf(%r13),%cl 0x0000000810f5c49e <+894>: mov %rax,0x3(%r13) 0x0000000810f5c4a2 <+898>: mov %cl,0x0(%r13) 0x0000000810f5c4a6 <+902>: mov 0x8(%r12),%rax 0x0000000810f5c4ab <+907>: mov 0x5(%r12),%cl 0x0000000810f5c4b0 <+912>: mov %cl,0x6(%rdx) 0x0000000810f5c4b3 <+915>: test %cl,%cl 0x0000000810f5c4b5 <+917>: jne 0x810f5c4ce <evalexpr_0_2+942> 0x0000000810f5c4b7 <+919>: cmpb $0x1,(%rax) 0x0000000810f5c4ba <+922>: jne 0x810f5c4ca <evalexpr_0_2+938> 0x0000000810f5c4bc <+924>: cmpb $0x3,0x1(%rax) 0x0000000810f5c4c0 <+928>: jne 0x810f5c4ca <evalexpr_0_2+938> 0x0000000810f5c4c2 <+930>: mov 0x2(%rax),%rax 0x0000000810f5c4c6 <+934>: add $0x22,%rax 0x0000000810f5c4ca <+938>: mov %rax,0x30(%rbx) 0x0000000810f5c4ce <+942>: mov (%r14),%rax 0x0000000810f5c4d1 <+945>: mov (%r15),%cl 0x0000000810f5c4d4 <+948>: mov %rax,0x1173(%r13) 0x0000000810f5c4db <+955>: mov %cl,0x117b(%r13) 0x0000000810f5c4e2 <+962>: movb $0x1,0x0(%r13) 0x0000000810f5c4e7 <+967>: cmp $0x1,%cl 0x0000000810f5c4ea <+970>: je 0x810f5c524 <evalexpr_0_2+1028> 0x0000000810f5c4ec <+972>: cmpb $0x1,0x118b(%r13) 0x0000000810f5c4f4 <+980>: je 0x810f5c524 <evalexpr_0_2+1028> 0x0000000810f5c4f6 <+982>: movb $0x0,0x116f(%r13) 0x0000000810f5c4fe <+990>: lea 0x1153(%r13),%rdi 0x0000000810f5c505 <+997>: movabs $0xaa0b40,%rax 0x0000000810f5c50f <+1007>: callq *%rax 0x0000000810f5c511 <+1009>: mov -0x30(%rbp),%rdx 0x0000000810f5c515 <+1013>: mov 0x116f(%r13),%cl 0x0000000810f5c51c <+1020>: mov %rax,0x3(%r13) 0x0000000810f5c520 <+1024>: mov %cl,0x0(%r13) 0x0000000810f5c524 <+1028>: mov 0x8(%r12),%rax 0x0000000810f5c529 <+1033>: mov 0x5(%r12),%cl 0x0000000810f5c52e <+1038>: mov %cl,0x7(%rdx) 0x0000000810f5c531 <+1041>: test %cl,%cl 0x0000000810f5c533 <+1043>: jne 0x810f5c54c <evalexpr_0_2+1068> 0x0000000810f5c535 <+1045>: cmpb $0x1,(%rax) 0x0000000810f5c538 <+1048>: jne 0x810f5c548 <evalexpr_0_2+1064> 0x0000000810f5c53a <+1050>: cmpb $0x3,0x1(%rax) 0x0000000810f5c53e <+1054>: jne 0x810f5c548 <evalexpr_0_2+1064> 0x0000000810f5c540 <+1056>: mov 0x2(%rax),%rax 0x0000000810f5c544 <+1060>: add $0x22,%rax 0x0000000810f5c548 <+1064>: mov %rax,0x38(%rbx) 0x0000000810f5c54c <+1068>: mov (%r14),%rax 0x0000000810f5c54f <+1071>: mov (%r15),%cl 0x0000000810f5c552 <+1074>: mov %rax,0x1213(%r13) 0x0000000810f5c559 <+1081>: mov %cl,0x121b(%r13) 0x0000000810f5c560 <+1088>: movb $0x1,0x0(%r13) 0x0000000810f5c565 <+1093>: cmp $0x1,%cl 0x0000000810f5c568 <+1096>: je 0x810f5c5a2 <evalexpr_0_2+1154> 0x0000000810f5c56a <+1098>: cmpb $0x1,0x122b(%r13) 0x0000000810f5c572 <+1106>: je 0x810f5c5a2 <evalexpr_0_2+1154> 0x0000000810f5c574 <+1108>: movb $0x0,0x120f(%r13) 0x0000000810f5c57c <+1116>: lea 0x11f3(%r13),%rdi 0x0000000810f5c583 <+1123>: movabs $0xaa0b40,%rax 0x0000000810f5c58d <+1133>: callq *%rax 0x0000000810f5c58f <+1135>: mov -0x30(%rbp),%rdx 0x0000000810f5c593 <+1139>: mov 0x120f(%r13),%cl 0x0000000810f5c59a <+1146>: mov %rax,0x3(%r13) 0x0000000810f5c59e <+1150>: mov %cl,0x0(%r13) 0x0000000810f5c5a2 <+1154>: mov 0x8(%r12),%rax 0x0000000810f5c5a7 <+1159>: mov 0x5(%r12),%cl 0x0000000810f5c5ac <+1164>: mov %cl,0x8(%rdx) 0x0000000810f5c5af <+1167>: test %cl,%cl 0x0000000810f5c5b1 <+1169>: jne 0x810f5c5ca <evalexpr_0_2+1194> 0x0000000810f5c5b3 <+1171>: cmpb $0x1,(%rax) 0x0000000810f5c5b6 <+1174>: jne 0x810f5c5c6 <evalexpr_0_2+1190> 0x0000000810f5c5b8 <+1176>: cmpb $0x3,0x1(%rax) 0x0000000810f5c5bc <+1180>: jne 0x810f5c5c6 <evalexpr_0_2+1190> 0x0000000810f5c5be <+1182>: mov 0x2(%rax),%rax 0x0000000810f5c5c2 <+1186>: add $0x22,%rax 0x0000000810f5c5c6 <+1190>: mov %rax,0x40(%rbx) 0x0000000810f5c5ca <+1194>: mov (%r14),%rax 0x0000000810f5c5cd <+1197>: mov (%r15),%cl 0x0000000810f5c5d0 <+1200>: mov %rax,0x12b3(%r13) 0x0000000810f5c5d7 <+1207>: mov %cl,0x12bb(%r13) 0x0000000810f5c5de <+1214>: movb $0x1,0x0(%r13) 0x0000000810f5c5e3 <+1219>: cmp $0x1,%cl 0x0000000810f5c5e6 <+1222>: je 0x810f5c620 <evalexpr_0_2+1280> 0x0000000810f5c5e8 <+1224>: cmpb $0x1,0x12cb(%r13) 0x0000000810f5c5f0 <+1232>: je 0x810f5c620 <evalexpr_0_2+1280> 0x0000000810f5c5f2 <+1234>: movb $0x0,0x12af(%r13) 0x0000000810f5c5fa <+1242>: lea 0x1293(%r13),%rdi 0x0000000810f5c601 <+1249>: movabs $0xaa0b40,%rax 0x0000000810f5c60b <+1259>: callq *%rax 0x0000000810f5c60d <+1261>: mov -0x30(%rbp),%rdx 0x0000000810f5c611 <+1265>: mov 0x12af(%r13),%cl 0x0000000810f5c618 <+1272>: mov %rax,0x3(%r13) 0x0000000810f5c61c <+1276>: mov %cl,0x0(%r13) 0x0000000810f5c620 <+1280>: mov 0x8(%r12),%rax 0x0000000810f5c625 <+1285>: mov 0x5(%r12),%cl 0x0000000810f5c62a <+1290>: mov %cl,0x9(%rdx) 0x0000000810f5c62d <+1293>: test %cl,%cl 0x0000000810f5c62f <+1295>: jne 0x810f5c648 <evalexpr_0_2+1320> 0x0000000810f5c631 <+1297>: cmpb $0x1,(%rax) 0x0000000810f5c634 <+1300>: jne 0x810f5c644 <evalexpr_0_2+1316> 0x0000000810f5c636 <+1302>: cmpb $0x3,0x1(%rax) 0x0000000810f5c63a <+1306>: jne 0x810f5c644 <evalexpr_0_2+1316> 0x0000000810f5c63c <+1308>: mov 0x2(%rax),%rax 0x0000000810f5c640 <+1312>: add $0x22,%rax 0x0000000810f5c644 <+1316>: mov %rax,0x48(%rbx) 0x0000000810f5c648 <+1320>: mov %rbx,-0x58(%rbp) 0x0000000810f5c64c <+1324>: mov (%r14),%rax 0x0000000810f5c64f <+1327>: mov (%r15),%cl 0x0000000810f5c652 <+1330>: mov %rax,0x1353(%r13) 0x0000000810f5c659 <+1337>: mov %cl,0x135b(%r13) 0x0000000810f5c660 <+1344>: movb $0x1,0x0(%r13) 0x0000000810f5c665 <+1349>: cmp $0x1,%cl 0x0000000810f5c668 <+1352>: je 0x810f5c7e3 <evalexpr_0_2+1731> 0x0000000810f5c66e <+1358>: cmpb $0x1,0x136b(%r13) 0x0000000810f5c676 <+1366>: je 0x810f5c7e3 <evalexpr_0_2+1731> 0x0000000810f5c67c <+1372>: movb $0x0,0x134f(%r13) 0x0000000810f5c684 <+1380>: lea 0x1333(%r13),%rdi 0x0000000810f5c68b <+1387>: movabs $0xaa0b40,%rax 0x0000000810f5c695 <+1397>: callq *%rax 0x0000000810f5c697 <+1399>: mov -0x30(%rbp),%rdx 0x0000000810f5c69b <+1403>: mov 0x134f(%r13),%bl 0x0000000810f5c6a2 <+1410>: mov %rax,0x3(%r13) 0x0000000810f5c6a6 <+1414>: mov %bl,0x0(%r13) 0x0000000810f5c6aa <+1418>: cmp $0x1,%bl 0x0000000810f5c6ad <+1421>: je 0x810f5c7e3 <evalexpr_0_2+1731> 0x0000000810f5c6b3 <+1427>: mov 0x3(%r13),%rax 0x0000000810f5c6b7 <+1431>: mov %rax,0x13f3(%r13) 0x0000000810f5c6be <+1438>: movb $0x0,0x13fb(%r13) 0x0000000810f5c6c6 <+1446>: lea 0x13d3(%r13),%rdi 0x0000000810f5c6cd <+1453>: movabs $0xb7cfc0,%rax 0x0000000810f5c6d7 <+1463>: callq *%rax 0x0000000810f5c6d9 <+1465>: mov -0x30(%rbp),%rdx 0x0000000810f5c6dd <+1469>: test %rax,%rax 0x0000000810f5c6e0 <+1472>: je 0x810f5c7e3 <evalexpr_0_2+1731> 0x0000000810f5c6e6 <+1478>: mov %rax,0x1493(%r13) 0x0000000810f5c6ed <+1485>: mov %bl,0x149b(%r13) 0x0000000810f5c6f4 <+1492>: movb $0x0,0x148f(%r13) 0x0000000810f5c6fc <+1500>: movzbl (%rax),%esi 0x0000000810f5c6ff <+1503>: movabs $0xc85a04,%rcx 0x0000000810f5c709 <+1513>: mov (%rcx),%edi 0x0000000810f5c70b <+1515>: mov 0x1b178bc(%rip),%rcx # 0x812a73fce 0x0000000810f5c712 <+1522>: mov %fs:0x0,%rdx 0x0000000810f5c71b <+1531>: mov (%rdx,%rcx,1),%rbx 0x0000000810f5c71f <+1535>: cmp %esi,%edi 0x0000000810f5c721 <+1537>: mov %rax,-0x60(%rbp) 0x0000000810f5c725 <+1541>: mov %edi,-0x48(%rbp) 0x0000000810f5c728 <+1544>: mov %rbx,-0x40(%rbp) 0x0000000810f5c72c <+1548>: jle 0x810f5c769 <evalexpr_0_2+1609> 0x0000000810f5c72e <+1550>: test %rbx,%rbx 0x0000000810f5c731 <+1553>: movabs $0xc85a10,%rcx 0x0000000810f5c73b <+1563>: mov (%rcx),%rdx 0x0000000810f5c73e <+1566>: mov %rbx,%rcx 0x0000000810f5c741 <+1569>: mov %rdx,%r8 0x0000000810f5c744 <+1572>: cmove %rdx,%rcx 0x0000000810f5c748 <+1576>: mov %rax,%rbx 0x0000000810f5c74b <+1579>: mov %rdx,-0x50(%rbp) 0x0000000810f5c74f <+1583>: nop 0x0000000810f5c750 <+1584>: movzbl %sil,%edx 0x0000000810f5c754 <+1588>: testb $0x40,0x41(%rcx,%rdx,8) 0x0000000810f5c759 <+1593>: je 0x810f5c77d <evalexpr_0_2+1629> 0x0000000810f5c75b <+1595>: movzbl 0x1(%rbx),%esi 0x0000000810f5c75f <+1599>: add $0x1,%rbx 0x0000000810f5c763 <+1603>: cmp %esi,%edi 0x0000000810f5c765 <+1605>: jg 0x810f5c750 <evalexpr_0_2+1584> 0x0000000810f5c767 <+1607>: jmp 0x810f5c77d <evalexpr_0_2+1629> 0x0000000810f5c769 <+1609>: movabs $0xc85a10,%rcx 0x0000000810f5c773 <+1619>: mov (%rcx),%rcx 0x0000000810f5c776 <+1622>: mov %rcx,-0x50(%rbp) 0x0000000810f5c77a <+1626>: mov %rax,%rbx 0x0000000810f5c77d <+1629>: movabs $0x801769f90,%rax 0x0000000810f5c787 <+1639>: mov %rbx,%rdi 0x0000000810f5c78a <+1642>: callq *%rax 0x0000000810f5c78c <+1644>: mov -0x40(%rbp),%rcx 0x0000000810f5c790 <+1648>: test %rcx,%rcx 0x0000000810f5c793 <+1651>: mov -0x50(%rbp),%rdi 0x0000000810f5c797 <+1655>: cmovne %rcx,%rdi 0x0000000810f5c79b <+1659>: mov -0x48(%rbp),%edx 0x0000000810f5c79e <+1662>: xchg %ax,%ax 0x0000000810f5c7a0 <+1664>: mov %rax,%rsi 0x0000000810f5c7a3 <+1667>: test %rax,%rax 0x0000000810f5c7a6 <+1670>: je 0x810f5c7bc <evalexpr_0_2+1692> 0x0000000810f5c7a8 <+1672>: movzbl -0x1(%rbx,%rsi,1),%ecx 0x0000000810f5c7ad <+1677>: cmp %ecx,%edx 0x0000000810f5c7af <+1679>: jle 0x810f5c7bc <evalexpr_0_2+1692> 0x0000000810f5c7b1 <+1681>: lea -0x1(%rsi),%rax 0x0000000810f5c7b5 <+1685>: testb $0x40,0x41(%rdi,%rcx,8) 0x0000000810f5c7ba <+1690>: jne 0x810f5c7a0 <evalexpr_0_2+1664> 0x0000000810f5c7bc <+1692>: movabs $0xa3acc0,%rax 0x0000000810f5c7c6 <+1702>: lea -0x32(%rbp),%rdx 0x0000000810f5c7ca <+1706>: mov %rbx,%rdi 0x0000000810f5c7cd <+1709>: callq *%rax 0x0000000810f5c7cf <+1711>: test %al,%al 0x0000000810f5c7d1 <+1713>: je 0x810f5c9cc <evalexpr_0_2+2220> 0x0000000810f5c7d7 <+1719>: movzbl -0x32(%rbp),%eax 0x0000000810f5c7db <+1723>: mov %rax,0x3(%r13) 0x0000000810f5c7df <+1727>: mov -0x30(%rbp),%rdx 0x0000000810f5c7e3 <+1731>: mov 0x8(%r12),%rax 0x0000000810f5c7e8 <+1736>: mov 0x5(%r12),%cl 0x0000000810f5c7ed <+1741>: mov %cl,0xa(%rdx) 0x0000000810f5c7f0 <+1744>: mov -0x58(%rbp),%rcx 0x0000000810f5c7f4 <+1748>: mov %rax,0x50(%rcx) 0x0000000810f5c7f8 <+1752>: mov (%r14),%rax 0x0000000810f5c7fb <+1755>: mov (%r15),%cl 0x0000000810f5c7fe <+1758>: mov %rax,0x31e763(%r13) 0x0000000810f5c805 <+1765>: mov %cl,0x31e76b(%r13) 0x0000000810f5c80c <+1772>: movb $0x1,0x0(%r13) 0x0000000810f5c811 <+1777>: cmp $0x1,%cl 0x0000000810f5c814 <+1780>: je 0x810f5c995 <evalexpr_0_2+2165> 0x0000000810f5c81a <+1786>: cmpb $0x1,0x31e77b(%r13) 0x0000000810f5c822 <+1794>: je 0x810f5c995 <evalexpr_0_2+2165> 0x0000000810f5c828 <+1800>: movb $0x0,0x31e75f(%r13) 0x0000000810f5c830 <+1808>: lea 0x31e743(%r13),%rdi 0x0000000810f5c837 <+1815>: movabs $0xaa0b40,%rax 0x0000000810f5c841 <+1825>: callq *%rax 0x0000000810f5c843 <+1827>: mov -0x30(%rbp),%rdx 0x0000000810f5c847 <+1831>: mov 0x31e75f(%r13),%bl 0x0000000810f5c84e <+1838>: mov %rax,0x3(%r13) 0x0000000810f5c852 <+1842>: mov %bl,0x0(%r13) 0x0000000810f5c856 <+1846>: cmp $0x1,%bl 0x0000000810f5c859 <+1849>: je 0x810f5c995 <evalexpr_0_2+2165> 0x0000000810f5c85f <+1855>: mov 0x3(%r13),%rax 0x0000000810f5c863 <+1859>: mov %rax,0x31e803(%r13) 0x0000000810f5c86a <+1866>: movb $0x0,0x31e80b(%r13) 0x0000000810f5c872 <+1874>: lea 0x31e7e3(%r13),%rdi 0x0000000810f5c879 <+1881>: movabs $0xb7cfc0,%rax 0x0000000810f5c883 <+1891>: callq *%rax 0x0000000810f5c885 <+1893>: mov -0x30(%rbp),%rdx 0x0000000810f5c889 <+1897>: test %rax,%rax 0x0000000810f5c88c <+1900>: je 0x810f5c995 <evalexpr_0_2+2165> 0x0000000810f5c892 <+1906>: mov %rax,0x1733(%r13) 0x0000000810f5c899 <+1913>: mov %bl,0x173b(%r13) 0x0000000810f5c8a0 <+1920>: movb $0x0,0x172f(%r13) 0x0000000810f5c8a8 <+1928>: movzbl (%rax),%esi 0x0000000810f5c8ab <+1931>: movabs $0xc85a04,%rcx 0x0000000810f5c8b5 <+1941>: mov (%rcx),%r14d 0x0000000810f5c8b8 <+1944>: mov 0x1b178bc(%rip),%rcx # 0x812a7417b 0x0000000810f5c8bf <+1951>: mov %fs:0x0,%rdx 0x0000000810f5c8c8 <+1960>: mov (%rdx,%rcx,1),%rdi 0x0000000810f5c8cc <+1964>: cmp %esi,%r14d 0x0000000810f5c8cf <+1967>: mov %rax,-0x40(%rbp) 0x0000000810f5c8d3 <+1971>: mov %rdi,-0x48(%rbp) 0x0000000810f5c8d7 <+1975>: jle 0x810f5c91a <evalexpr_0_2+2042> 0x0000000810f5c8d9 <+1977>: test %rdi,%rdi 0x0000000810f5c8dc <+1980>: movabs $0xc85a10,%rcx 0x0000000810f5c8e6 <+1990>: mov (%rcx),%r15 0x0000000810f5c8e9 <+1993>: mov %rdi,%rcx 0x0000000810f5c8ec <+1996>: cmove %r15,%rcx 0x0000000810f5c8f0 <+2000>: mov %rax,%rbx 0x0000000810f5c8f3 <+2003>: nopw %cs:0x0(%rax,%rax,1) 0x0000000810f5c8fd <+2013>: nopl (%rax) 0x0000000810f5c900 <+2016>: movzbl %sil,%edx 0x0000000810f5c904 <+2020>: testb $0x40,0x41(%rcx,%rdx,8) 0x0000000810f5c909 <+2025>: je 0x810f5c92a <evalexpr_0_2+2058> 0x0000000810f5c90b <+2027>: movzbl 0x1(%rbx),%esi 0x0000000810f5c90f <+2031>: add $0x1,%rbx 0x0000000810f5c913 <+2035>: cmp %esi,%r14d 0x0000000810f5c916 <+2038>: jg 0x810f5c900 <evalexpr_0_2+2016> 0x0000000810f5c918 <+2040>: jmp 0x810f5c92a <evalexpr_0_2+2058> 0x0000000810f5c91a <+2042>: movabs $0xc85a10,%rcx 0x0000000810f5c924 <+2052>: mov (%rcx),%r15 0x0000000810f5c927 <+2055>: mov %rax,%rbx 0x0000000810f5c92a <+2058>: movabs $0x801769f90,%rax 0x0000000810f5c934 <+2068>: mov %rbx,%rdi 0x0000000810f5c937 <+2071>: callq *%rax 0x0000000810f5c939 <+2073>: mov -0x48(%rbp),%rcx 0x0000000810f5c93d <+2077>: test %rcx,%rcx 0x0000000810f5c940 <+2080>: cmovne %rcx,%r15 0x0000000810f5c944 <+2084>: nopw %cs:0x0(%rax,%rax,1) 0x0000000810f5c94e <+2094>: xchg %ax,%ax 0x0000000810f5c950 <+2096>: mov %rax,%rsi 0x0000000810f5c953 <+2099>: test %rax,%rax 0x0000000810f5c956 <+2102>: je 0x810f5c96e <evalexpr_0_2+2126> 0x0000000810f5c958 <+2104>: movzbl -0x1(%rbx,%rsi,1),%ecx 0x0000000810f5c95d <+2109>: cmp %ecx,%r14d 0x0000000810f5c960 <+2112>: jle 0x810f5c96e <evalexpr_0_2+2126> 0x0000000810f5c962 <+2114>: lea -0x1(%rsi),%rax 0x0000000810f5c966 <+2118>: testb $0x40,0x41(%r15,%rcx,8) 0x0000000810f5c96c <+2124>: jne 0x810f5c950 <evalexpr_0_2+2096> 0x0000000810f5c96e <+2126>: movabs $0xa3acc0,%rax 0x0000000810f5c978 <+2136>: lea -0x31(%rbp),%rdx 0x0000000810f5c97c <+2140>: mov %rbx,%rdi 0x0000000810f5c97f <+2143>: callq *%rax 0x0000000810f5c981 <+2145>: test %al,%al 0x0000000810f5c983 <+2147>: je 0x810f5ca14 <evalexpr_0_2+2292> 0x0000000810f5c989 <+2153>: movzbl -0x31(%rbp),%eax 0x0000000810f5c98d <+2157>: mov %rax,0x3(%r13) 0x0000000810f5c991 <+2161>: mov -0x30(%rbp),%rdx 0x0000000810f5c995 <+2165>: mov 0x8(%r12),%rax 0x0000000810f5c99a <+2170>: mov 0x5(%r12),%cl 0x0000000810f5c99f <+2175>: mov %cl,0xb(%rdx) 0x0000000810f5c9a2 <+2178>: mov -0x58(%rbp),%rcx 0x0000000810f5c9a6 <+2182>: mov %rax,0x58(%rcx) 0x0000000810f5c9aa <+2186>: mov 0x8(%r12),%rax 0x0000000810f5c9af <+2191>: mov 0x5(%r12),%cl 0x0000000810f5c9b4 <+2196>: and $0x1,%cl 0x0000000810f5c9b7 <+2199>: mov -0x68(%rbp),%rdx 0x0000000810f5c9bb <+2203>: mov %cl,(%rdx) 0x0000000810f5c9bd <+2205>: add $0x48,%rsp 0x0000000810f5c9c1 <+2209>: pop %rbx 0x0000000810f5c9c2 <+2210>: pop %r12 0x0000000810f5c9c4 <+2212>: pop %r13 0x0000000810f5c9c6 <+2214>: pop %r14 0x0000000810f5c9c8 <+2216>: pop %r15 0x0000000810f5c9ca <+2218>: pop %rbp 0x0000000810f5c9cb <+2219>: retq 0x0000000810f5c9cc <+2220>: movabs $0xbbf270,%rax 0x0000000810f5c9d6 <+2230>: mov $0x14,%edi 0x0000000810f5c9db <+2235>: xor %esi,%esi 0x0000000810f5c9dd <+2237>: callq *%rax 0x0000000810f5c9df <+2239>: movabs $0xbc0510,%rax 0x0000000810f5c9e9 <+2249>: mov $0x2020082,%edi 0x0000000810f5c9ee <+2254>: callq *%rax 0x0000000810f5c9f0 <+2256>: movabs $0x810f5d023,%rdi 0x0000000810f5c9fa <+2266>: movabs $0x810f5d04a,%rsi 0x0000000810f5ca04 <+2276>: movabs $0xbc0800,%rcx 0x0000000810f5ca0e <+2286>: mov -0x60(%rbp),%rdx 0x0000000810f5ca12 <+2290>: jmp 0x810f5ca5a <evalexpr_0_2+2362> 0x0000000810f5ca14 <+2292>: movabs $0xbbf270,%rax 0x0000000810f5ca1e <+2302>: mov $0x14,%edi 0x0000000810f5ca23 <+2307>: xor %esi,%esi 0x0000000810f5ca25 <+2309>: callq *%rax 0x0000000810f5ca27 <+2311>: movabs $0xbc0510,%rax 0x0000000810f5ca31 <+2321>: mov $0x2020082,%edi 0x0000000810f5ca36 <+2326>: callq *%rax 0x0000000810f5ca38 <+2328>: movabs $0x810f5d023,%rdi 0x0000000810f5ca42 <+2338>: movabs $0x810f5d04a,%rsi 0x0000000810f5ca4c <+2348>: movabs $0xbc0800,%rcx 0x0000000810f5ca56 <+2358>: mov -0x40(%rbp),%rdx 0x0000000810f5ca5a <+2362>: xor %eax,%eax 0x0000000810f5ca5c <+2364>: callq *%rcx 0x0000000810f5ca5e <+2366>: movabs $0x810f5d052,%rdi 0x0000000810f5ca68 <+2376>: movabs $0x810f5d059,%rdx 0x0000000810f5ca72 <+2386>: movabs $0xbbfc50,%rax 0x0000000810f5ca7c <+2396>: mov $0x9a,%esi 0x0000000810f5ca81 <+2401>: callq *%rax End of assembler dump. -- Dmitry Marakasov . 55B5 0596 FF1E 8D84 5F56 9510 D35A 80DD F9D2 F77D amdmi3@amdmi3.ru ..: https://github.com/AMDmi3 -
Re: BUG #16696: Backend crash in llvmjit
Dmitry Marakasov <amdmi3@amdmi3.ru> — 2020-11-04T21:20:15Z
* Andres Freund (andres@anarazel.de) wrote: > > > Environment details: > > > - FreeBSD 12.1 amd64 > > > - PostgreSQL 13.0 (built from FreeBSD ports) > > > - llvm-10.0.1 (build from FreeBSD ports) > > > > My bad, it's actually llvm-9.0.1. Multiple llvm versions are installed on > > the system, and PostgreSQL uses llvm9: > > > > ldd /usr/local/lib/postgresql/llvmjit.so | grep LLVM > > libLLVM-9.so => /usr/local/llvm90/lib/libLLVM-9.so (0x800e00000) > > Could you try generating a backtrace after turning jit_debugging_support on? That might give a bit more information. > > I'll check once I'm home whether I can reproduce in my environment. I did some digging. First of all, I've discovered that the problem goes away if llvm bitcode optimization is disabled (by commenting out llvm_optimize_module call). I've dumped the opcode and tried compiling it back to match disassembly of the failing function in gdb disassembly. It didn't match perfectly, but this place looked similar: # %bb.84: # %op.32.inputcall movq %rax, 5267(%r13) movb %bl, 5275(%r13) movb $0, 5263(%r13) movzbl (%rax), %esi movl __mb_sb_limit(%rip), %edi movq _ThreadRuneLocale@GOTTPOFF(%rip), %rcx movq %fs:0, %rdx movq (%rdx,%rcx), %rcx cmpl %esi, %edi movq %rax, -96(%rbp) # 8-byte Spill movl %edi, -72(%rbp) # 4-byte Spill movq %rcx, -64(%rbp) # 8-byte Spill jle .LBB1_85 Here's my hypothesis: The problem happens when boolin() function is inlined by LLVM. The named function calls isspace() internally, which on FreeBSD is locale-specific and involves caching some locale parameters in thread-local variable defined as extern _Thread_local const _RuneLocale *_ThreadRuneLocale; The execution crashes on trying to access the named thread-local varible, probably because something related to TLS is not set up properly in/for LLVM. I've confirmed this hypothesis by disabling isspace() calls in boolin() which has also fixed the problem. -- Dmitry Marakasov . 55B5 0596 FF1E 8D84 5F56 9510 D35A 80DD F9D2 F77D amdmi3@amdmi3.ru ..: https://github.com/AMDmi3
-
Re: BUG #16696: Backend crash in llvmjit
Dmitry Marakasov <amdmi3@amdmi3.ru> — 2020-11-04T23:50:54Z
* Dmitry Marakasov (amdmi3@amdmi3.ru) wrote: > > > > Environment details: > > > > - FreeBSD 12.1 amd64 > > > > - PostgreSQL 13.0 (built from FreeBSD ports) > > > > - llvm-10.0.1 (build from FreeBSD ports) > > > > > > My bad, it's actually llvm-9.0.1. Multiple llvm versions are installed on > > > the system, and PostgreSQL uses llvm9: > > > > > > ldd /usr/local/lib/postgresql/llvmjit.so | grep LLVM > > > libLLVM-9.so => /usr/local/llvm90/lib/libLLVM-9.so (0x800e00000) > > > > Could you try generating a backtrace after turning jit_debugging_support on? That might give a bit more information. > > > > I'll check once I'm home whether I can reproduce in my environment. > > I did some digging. First of all, I've discovered that the problem > goes away if llvm bitcode optimization is disabled (by commenting out > llvm_optimize_module call). > > I've dumped the opcode and tried compiling it back to match disassembly > of the failing function in gdb disassembly. It didn't match perfectly, > but this place looked similar: > > # %bb.84: # %op.32.inputcall > movq %rax, 5267(%r13) > movb %bl, 5275(%r13) > movb $0, 5263(%r13) > movzbl (%rax), %esi > movl __mb_sb_limit(%rip), %edi > movq _ThreadRuneLocale@GOTTPOFF(%rip), %rcx > movq %fs:0, %rdx > movq (%rdx,%rcx), %rcx > cmpl %esi, %edi > movq %rax, -96(%rbp) # 8-byte Spill > movl %edi, -72(%rbp) # 4-byte Spill > movq %rcx, -64(%rbp) # 8-byte Spill > jle .LBB1_85 > > Here's my hypothesis: > > The problem happens when boolin() function is inlined by LLVM. > The named function calls isspace() internally, which on FreeBSD is > locale-specific and involves caching some locale parameters in > thread-local variable defined as > > extern _Thread_local const _RuneLocale *_ThreadRuneLocale; > > The execution crashes on trying to access the named thread-local varible, > probably because something related to TLS is not set up properly in/for > LLVM. > > I've confirmed this hypothesis by disabling isspace() calls in boolin() > which has also fixed the problem. Long story short, I was able to mitigate the crash with the following patch: --- disable-inlining-tls-using-functions.patch begins here --- commit f703544edc406293e39b7a59a245e798d18f458e Author: Dmitry Marakasov <amdmi3@amdmi3.ru> Date: Thu Nov 5 02:56:00 2020 +0300 Do not inline functions accessing TLS in LLVM JIT diff --git src/backend/jit/llvm/llvmjit_inline.cpp src/backend/jit/llvm/llvmjit_inline.cpp index 2617a46..a063edb 100644 --- src/backend/jit/llvm/llvmjit_inline.cpp +++ src/backend/jit/llvm/llvmjit_inline.cpp @@ -608,6 +608,16 @@ function_inlinable(llvm::Function &F, if (rv->materialize()) elog(FATAL, "failed to materialize metadata"); + /* + * Don't inline functions with thread-local variables until + * related crashes are investigated (see BUG #16696) + */ + if (rv->isThreadLocal()) { + ilog(DEBUG1, "cannot inline %s due to thread-local variable %s", + F.getName().data(), rv->getName().data()); + return false; + } + /* * Never want to inline externally visible vars, cheap enough to * reference. --- disable-inlining-tls-using-functions.patch ends here --- I have no knowledge of LLVM to investigate this further, but the guess is that something TLS related is not initialized properly. -- Dmitry Marakasov . 55B5 0596 FF1E 8D84 5F56 9510 D35A 80DD F9D2 F77D amdmi3@amdmi3.ru ..: https://github.com/AMDmi3 -
Re: BUG #16696: Backend crash in llvmjit
Andres Freund <andres@anarazel.de> — 2020-11-10T05:48:02Z
Hi, On 2020-11-05 00:20:15 +0300, Dmitry Marakasov wrote: > The problem happens when boolin() function is inlined by LLVM. > The named function calls isspace() internally, which on FreeBSD is > locale-specific and involves caching some locale parameters in > thread-local variable defined as > > extern _Thread_local const _RuneLocale *_ThreadRuneLocale; > > The execution crashes on trying to access the named thread-local varible, > probably because something related to TLS is not set up properly in/for > LLVM. > > I've confirmed this hypothesis by disabling isspace() calls in boolin() > which has also fixed the problem. Nice digging. I suspect your approach of disabling inlining for TLS might be the most reasonable one for now. Any chance you're interested in extending your patch with a testcase that crashes reliably without it? Best on multiple platforms, including linux? Easiest way would probably be to add a function referencing a thread local variable in regress.c and set jit_above_cost = 0, jit_inline_above_cost=0 for a query using that function. Greetings, Andres Freund
-
Re: BUG #16696: Backend crash in llvmjit
Dmitry Marakasov <amdmi3@amdmi3.ru> — 2020-11-16T00:45:29Z
* Andres Freund (andres@anarazel.de) wrote: > > The problem happens when boolin() function is inlined by LLVM. > > The named function calls isspace() internally, which on FreeBSD is > > locale-specific and involves caching some locale parameters in > > thread-local variable defined as > > > > extern _Thread_local const _RuneLocale *_ThreadRuneLocale; > > > > The execution crashes on trying to access the named thread-local varible, > > probably because something related to TLS is not set up properly in/for > > LLVM. > > > > I've confirmed this hypothesis by disabling isspace() calls in boolin() > > which has also fixed the problem. > > Nice digging. I suspect your approach of disabling inlining for TLS > might be the most reasonable one for now. > > Any chance you're interested in extending your patch with a testcase > that crashes reliably without it? Best on multiple platforms, including > linux? Easiest way would probably be to add a function referencing a > thread local variable in regress.c and set jit_above_cost = 0, > jit_inline_above_cost=0 for a query using that function. Can try. Here's what I could come with so far: https://github.com/postgres/postgres/compare/master...AMDmi3:llvm-jit-regress.patch Unfortunately it doesn't crash with the dedicated function, likely because there's no llvm bitcode generation performed for regress.c. It does however reliably crash on SELECT (jsonb_array_elements('[true]'::jsonb)->>0)::boolean; or SELECT (case when random() >= 0.5 then 'true' else 'false' end)::boolean; but it would likely not reproduce on Linux due to different isspace implementation. Notably, it does not crash on simple 'true'::boolean, I guess because it may be evaluated at compile time. -- Dmitry Marakasov . 55B5 0596 FF1E 8D84 5F56 9510 D35A 80DD F9D2 F77D amdmi3@amdmi3.ru ..: https://github.com/AMDmi3 -
Re: BUG #16696: Backend crash in llvmjit
Palle Girgensohn <girgen@pingpong.net> — 2021-05-19T14:49:02Z
Hi, This seems to still not be fixed. Is it planned for 14.0 release? Palle -- Sent from: https://www.postgresql-archive.org/PostgreSQL-bugs-f2117394.html
-
Re: BUG #16696: Backend crash in llvmjit
Thomas Munro <thomas.munro@gmail.com> — 2021-07-18T05:13:50Z
On Thu, May 20, 2021 at 2:49 AM Palle <girgen@pingpong.net> wrote: > This seems to still not be fixed. Is it planned for 14.0 release? I don't know enough about LLVM and TLS to have an opinion on the details, but I tested this and it works, and I think it should probably be back-patched based on the discussion so far (and hopefully in the future some way can be figured out to make it actually work?). When I tested the original repro, I got SIGTRAP (not SIGSEGV as reported) with LLVM 9 and 10, while 11, 12 and 13 failed gracefully with: 2021-07-18 15:27:42.785 NZST [78884] FATAL: fatal llvm error: Relocation type not implemented yet! I was reminded of this thread because I happened to see a relevant-looking commit whiz past in the FreeBSD commit log[1]: libc is changing to a different TLS model. I just tried that, and it makes no difference. In case it helps someone who wants to see this failure, I recently XKCD1319'd myself into producing Vagrant files for PostgreSQL hacking on various OSes[2] after a clash with illumos on the build farm (cd freebsd12; vagrant up; vagrant ssh; you'll find a fresh build of the PostgreSQL master branch built against LLVM 9 under ~/install). [1] https://cgit.freebsd.org/src/commit/?id=9c97062b620137a1f7cad4c6b3fb030a396b3266 [2] https://github.com/macdice/postgresql-dev-vagrant/
-
Re: BUG #16696: Backend crash in llvmjit
Palle Girgensohn <girgen@pingpong.se> — 2021-07-21T09:31:11Z
Hi! I’m on holiday and AFK another week or so. Please feel free to commit if it seems correct. Palle > 18 juli 2021 kl. 07:14 skrev Thomas Munro <thomas.munro@gmail.com>: > > On Thu, May 20, 2021 at 2:49 AM Palle <girgen@pingpong.net> wrote: >> This seems to still not be fixed. Is it planned for 14.0 release? > > I don't know enough about LLVM and TLS to have an opinion on the > details, but I tested this and it works, and I think it should > probably be back-patched based on the discussion so far (and hopefully > in the future some way can be figured out to make it actually work?). > > When I tested the original repro, I got SIGTRAP (not SIGSEGV as > reported) with LLVM 9 and 10, while 11, 12 and 13 failed gracefully > with: > > 2021-07-18 15:27:42.785 NZST [78884] FATAL: fatal llvm error: > Relocation type not implemented yet! > > I was reminded of this thread because I happened to see a > relevant-looking commit whiz past in the FreeBSD commit log[1]: libc > is changing to a different TLS model. I just tried that, and it makes > no difference. > > In case it helps someone who wants to see this failure, I recently > XKCD1319'd myself into producing Vagrant files for PostgreSQL hacking > on various OSes[2] after a clash with illumos on the build farm (cd > freebsd12; vagrant up; vagrant ssh; you'll find a fresh build of the > PostgreSQL master branch built against LLVM 9 under ~/install). > > [1] https://cgit.freebsd.org/src/commit/?id=9c97062b620137a1f7cad4c6b3fb030a396b3266 > [2] https://github.com/macdice/postgresql-dev-vagrant/
-
Re: BUG #16696: Backend crash in llvmjit
Andres Freund <andres@anarazel.de> — 2021-07-21T16:59:27Z
Hi, On 2021-07-18 17:13:50 +1200, Thomas Munro wrote: > On Thu, May 20, 2021 at 2:49 AM Palle <girgen@pingpong.net> wrote: > > This seems to still not be fixed. Is it planned for 14.0 release? > > I don't know enough about LLVM and TLS to have an opinion on the > details, but I tested this and it works, and I think it should > probably be back-patched based on the discussion so far (and hopefully > in the future some way can be figured out to make it actually work?). > > When I tested the original repro, I got SIGTRAP (not SIGSEGV as > reported) with LLVM 9 and 10, while 11, 12 and 13 failed gracefully > with: > > 2021-07-18 15:27:42.785 NZST [78884] FATAL: fatal llvm error: > Relocation type not implemented yet! > > I was reminded of this thread because I happened to see a > relevant-looking commit whiz past in the FreeBSD commit log[1]: libc > is changing to a different TLS model. I just tried that, and it makes > no difference. FWIW, there's ongoing work in llvm to make the jit stuff handle TLS correctly. Greetings, Andres Freund
-
Re: BUG #16696: Backend crash in llvmjit
Thomas Munro <thomas.munro@gmail.com> — 2021-07-22T03:30:32Z
On Thu, Jul 22, 2021 at 4:59 AM Andres Freund <andres@anarazel.de> wrote: > On 2021-07-18 17:13:50 +1200, Thomas Munro wrote: > > On Thu, May 20, 2021 at 2:49 AM Palle <girgen@pingpong.net> wrote: > > > This seems to still not be fixed. Is it planned for 14.0 release? > > > > I don't know enough about LLVM and TLS to have an opinion on the > > details, but I tested this and it works, and I think it should > > probably be back-patched based on the discussion so far (and hopefully > > in the future some way can be figured out to make it actually work?). Done. For the record, the original repro now causes PostgreSQL on FreeBSD (with INLINE_DEBUG defined) to say: 2021-07-22 15:20:51.738 NZST [82982] DEBUG: checking inlinability of boolin 2021-07-22 15:20:51.738 NZST [82982] DEBUG: cannot inline boolin due to thread-local variable _ThreadRuneLocale 2021-07-22 15:20:51.738 NZST [82982] DEBUG: had to skip inlining boolin > FWIW, there's ongoing work in llvm to make the jit stuff handle TLS > correctly. Cool. Dmitry: Thanks very much for the report, reproducer, analysis and patch!
-
Re: BUG #16696: Backend crash in llvmjit
Dmitry Marakasov <amdmi3@amdmi3.ru> — 2021-07-22T14:06:34Z
* Thomas Munro (thomas.munro@gmail.com) wrote: > On Thu, Jul 22, 2021 at 4:59 AM Andres Freund <andres@anarazel.de> wrote: > > On 2021-07-18 17:13:50 +1200, Thomas Munro wrote: > > > On Thu, May 20, 2021 at 2:49 AM Palle <girgen@pingpong.net> wrote: > > > > This seems to still not be fixed. Is it planned for 14.0 release? > > > > > > I don't know enough about LLVM and TLS to have an opinion on the > > > details, but I tested this and it works, and I think it should > > > probably be back-patched based on the discussion so far (and hopefully > > > in the future some way can be figured out to make it actually work?). > > Done. > > For the record, the original repro now causes PostgreSQL on FreeBSD > (with INLINE_DEBUG defined) to say: > > 2021-07-22 15:20:51.738 NZST [82982] DEBUG: checking inlinability of boolin > 2021-07-22 15:20:51.738 NZST [82982] DEBUG: cannot inline boolin due > to thread-local variable _ThreadRuneLocale > 2021-07-22 15:20:51.738 NZST [82982] DEBUG: had to skip inlining boolin > > > FWIW, there's ongoing work in llvm to make the jit stuff handle TLS > > correctly. > > Cool. > > Dmitry: Thanks very much for the report, reproducer, analysis and patch! Superb, thanks a lot! -- Dmitry Marakasov . 55B5 0596 FF1E 8D84 5F56 9510 D35A 80DD F9D2 F77D amdmi3@amdmi3.ru ..: https://github.com/AMDmi3 https://amdmi3.ru