Re: pg11.1 jit segv

Justin Pryzby <pryzby@telsasoft.com>

From: Justin Pryzby <pryzby@telsasoft.com>
To: Andres Freund <andres@anarazel.de>
Cc: pgsql-hackers@postgresql.org
Date: 2018-11-15T23:03:35Z
Lists: pgsql-hackers
On Thu, Nov 15, 2018 at 02:47:55PM -0800, Andres Freund wrote:
> > (gdb) bt
> > #0  0x00007f08127e814e in ?? ()
> > #1  0x0000000000000000 in ?? ()
> 
> Could you enable jit_debugging_support and reproduce?  That should give
> a more useful backtrace.

Core was generated by `postgres: pryzbyj ts [local] EXPLAIN                   '.
Program terminated with signal 11, Segmentation fault.
#0  0x00007f819e227cb9 in __memcpy_ssse3_back () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install audit-libs-2.8.1-3.el7_5.1.x86_64 bzip2-libs-1.0.6-13.el7.x86_64 cyrus-sasl-lib-2.1.26-23.el7.x86_64 elfutils-libelf-0.170-4.el7.x86_64 elfutils-libs-0.170-4.el7.x86_64 glib4
(gdb) bt
#0  0x00007f819e227cb9 in __memcpy_ssse3_back () from /lib64/libc.so.6
#1  0x0000000000824076 in memcpy (__len=18446744073709551612, __src=0x6c35818, __dest=<optimized out>) at /usr/include/bits/string3.h:51
#2  varstrfastcmp_locale (x=113465364, y=113465636, ssup=<optimized out>) at varlena.c:2135
#3  0x000000000087c6f4 in ApplySortComparator (ssup=0x6c33c40, isNull2=<optimized out>, datum2=<optimized out>, isNull1=false, datum1=<optimized out>) at ../../../../src/include/utils/sortsupport.h:224
#4  comparetup_heap (a=<optimized out>, b=<optimized out>, state=0x6c337f0) at tuplesort.c:3560
#5  0x00000000008793f5 in qsort_tuple (a=0x8311958, n=5272, cmp_tuple=0x87c2d0 <comparetup_heap>, state=state@entry=0x6c337f0) at qsort_tuple.c:112
#6  0x000000000087d2db in tuplesort_sort_memtuples (state=state@entry=0x6c337f0) at tuplesort.c:3320
#7  0x000000000087ee54 in tuplesort_performsort (state=state@entry=0x6c337f0) at tuplesort.c:1811
#8  0x0000000000628ce3 in ExecSort (pstate=0x5e50ed8) at nodeSort.c:118
#9  0x0000000000609068 in ExecProcNodeInstr (node=0x5e50ed8) at execProcnode.c:461
#10 0x0000000000610d49 in ExecProcNode (node=0x5e50ed8) at ../../../src/include/executor/executor.h:237
#11 fetch_input_tuple (aggstate=aggstate@entry=0x5e50aa0) at nodeAgg.c:406
#12 0x0000000000612780 in agg_retrieve_direct (aggstate=0x5e50aa0) at nodeAgg.c:1736
#13 ExecAgg (pstate=0x5e50aa0) at nodeAgg.c:1551
#14 0x0000000000609068 in ExecProcNodeInstr (node=0x5e50aa0) at execProcnode.c:461
#15 0x000000000060a765 in ExecScanFetch (recheckMtd=0x62be50 <SubqueryRecheck>, accessMtd=0x62be70 <SubqueryNext>, node=0x5e508e0) at execScan.c:95
#16 ExecScan (node=0x5e508e0, accessMtd=0x62be70 <SubqueryNext>, recheckMtd=0x62be50 <SubqueryRecheck>) at execScan.c:162
#17 0x0000000000609068 in ExecProcNodeInstr (node=0x5e508e0) at execProcnode.c:461
#18 0x000000000061b365 in ExecProcNode (node=0x5e508e0) at ../../../src/include/executor/executor.h:237
#19 MultiExecPrivateHash (node=0x5e50720) at nodeHash.c:164
#20 MultiExecHash (node=node@entry=0x5e50720) at nodeHash.c:114
#21 0x0000000000609610 in MultiExecProcNode (node=node@entry=0x5e50720) at execProcnode.c:501
#22 0x000000000061bd98 in ExecHashJoinImpl (parallel=false, pstate=0x5e4f620) at nodeHashjoin.c:290
#23 ExecHashJoin (pstate=0x5e4f620) at nodeHashjoin.c:565
#24 0x0000000000609068 in ExecProcNodeInstr (node=0x5e4f620) at execProcnode.c:461
#25 0x000000000061b365 in ExecProcNode (node=0x5e4f620) at ../../../src/include/executor/executor.h:237
#26 MultiExecPrivateHash (node=0x5e4f460) at nodeHash.c:164
#27 MultiExecHash (node=node@entry=0x5e4f460) at nodeHash.c:114
#28 0x0000000000609610 in MultiExecProcNode (node=node@entry=0x5e4f460) at execProcnode.c:501
#29 0x000000000061bd98 in ExecHashJoinImpl (parallel=false, pstate=0x38344f8) at nodeHashjoin.c:290
#30 ExecHashJoin (pstate=0x38344f8) at nodeHashjoin.c:565
#31 0x0000000000609068 in ExecProcNodeInstr (node=0x38344f8) at execProcnode.c:461
#32 0x000000000061b365 in ExecProcNode (node=0x38344f8) at ../../../src/include/executor/executor.h:237
#33 MultiExecPrivateHash (node=0x38343c8) at nodeHash.c:164
#34 MultiExecHash (node=node@entry=0x38343c8) at nodeHash.c:114
#35 0x0000000000609610 in MultiExecProcNode (node=node@entry=0x38343c8) at execProcnode.c:501
#36 0x000000000061bd98 in ExecHashJoinImpl (parallel=false, pstate=0x3833148) at nodeHashjoin.c:290
#37 ExecHashJoin (pstate=0x3833148) at nodeHashjoin.c:565
#38 0x0000000000609068 in ExecProcNodeInstr (node=0x3833148) at execProcnode.c:461
#39 0x0000000000628cd6 in ExecProcNode (node=0x3833148) at ../../../src/include/executor/executor.h:237
#40 ExecSort (pstate=0x3832bd0) at nodeSort.c:107
#41 0x0000000000609068 in ExecProcNodeInstr (node=0x3832bd0) at execProcnode.c:461
#42 0x0000000000610d49 in ExecProcNode (node=0x3832bd0) at ../../../src/include/executor/executor.h:237
#43 fetch_input_tuple (aggstate=aggstate@entry=0x3832e30) at nodeAgg.c:406
#44 0x0000000000612780 in agg_retrieve_direct (aggstate=0x3832e30) at nodeAgg.c:1736
#45 ExecAgg (pstate=0x3832e30) at nodeAgg.c:1551
#46 0x0000000000609068 in ExecProcNodeInstr (node=0x3832e30) at execProcnode.c:461
#47 0x00000000006025ba in ExecProcNode (node=0x3832e30) at ../../../src/include/executor/executor.h:237
#48 ExecutePlan (execute_once=<optimized out>, dest=0xced3a0 <donothingDR>, direction=<optimized out>, numberTuples=0, sendTuples=true, operation=CMD_SELECT, use_parallel_mode=<optimized out>, planstate=0x3832e30,
    estate=0x3832970) at execMain.c:1723
#49 standard_ExecutorRun (queryDesc=0x5c3ec68, direction=<optimized out>, count=0, execute_once=<optimized out>) at execMain.c:364
#50 0x00007f8191ade08d in pgss_ExecutorRun (queryDesc=0x5c3ec68, direction=ForwardScanDirection, count=0, execute_once=<optimized out>) at pg_stat_statements.c:892
#51 0x00007f80eb8bf5e1 in explain_ExecutorRun (queryDesc=0x5c3ec68, direction=ForwardScanDirection, count=0, execute_once=<optimized out>) at auto_explain.c:268
#52 0x00000000005a497c in ExplainOnePlan (plannedstmt=plannedstmt@entry=0x5c32cd8, into=into@entry=0x0, es=es@entry=0x207aaf8,
    queryString=queryString@entry=0x1ec5d58 "explain(analyze)\nSELECT site_office as site_gran, site_location AS bs, sect_mscid || ' (' || gsm_site_name_to_sect_name (site_name, sect_num, sect_name) || ')' AS sitename, data_u5
#53 0x00000000005a4c87 in ExplainOneQuery (query=<optimized out>, cursorOptions=<optimized out>, into=0x0, es=0x207aaf8,
    queryString=0x1ec5d58 "explain(analyze)\nSELECT site_office as site_gran, site_location AS bs, sect_mscid || ' (' || gsm_site_name_to_sect_name (site_name, sect_num, sect_name) || ')' AS sitename, data_utrancell.period as1
#54 0x00000000005a5174 in ExplainQuery (pstate=pstate@entry=0x2078dc8, stmt=stmt@entry=0x1fe1df8,
    queryString=queryString@entry=0x1ec5d58 "explain(analyze)\nSELECT site_office as site_gran, site_location AS bs, sect_mscid || ' (' || gsm_site_name_to_sect_name (site_name, sect_num, sect_name) || ')' AS sitename, data_u4
#55 0x00000000007469dd in standard_ProcessUtility (pstmt=pstmt@entry=0x1fe1ea8,
    queryString=queryString@entry=0x1ec5d58 "explain(analyze)\nSELECT site_office as site_gran, site_location AS bs, sect_mscid || ' (' || gsm_site_name_to_sect_name (site_name, sect_num, sect_name) || ')' AS sitename, data_u5
#56 0x00007f8191ae0025 in pgss_ProcessUtility (pstmt=0x1fe1ea8,
    queryString=0x1ec5d58 "explain(analyze)\nSELECT site_office as site_gran, site_location AS bs, sect_mscid || ' (' || gsm_site_name_to_sect_name (site_name, sect_num, sect_name) || ')' AS sitename, data_utrancell.period as5
#57 0x0000000000743e86 in PortalRunUtility (portal=0x1f12be0, pstmt=0x1fe1ea8, isTopLevel=<optimized out>, setHoldSnapshot=<optimized out>, dest=0x2078d38, completionTag=0x7ffc304d0e20 "") at pquery.c:1178
#58 0x0000000000744bc2 in FillPortalStore (portal=portal@entry=0x1f12be0, isTopLevel=isTopLevel@entry=true) at pquery.c:1038
#59 0x000000000074564f in PortalRun (portal=<optimized out>, count=9223372036854775807, isTopLevel=<optimized out>, run_once=<optimized out>, dest=0x7f81a09e2dc8, altdest=0x7f81a09e2dc8, completionTag=0x7ffc304d1040 "")
    at pquery.c:768
#60 0x0000000000741577 in exec_simple_query (query_string=<optimized out>) at postgres.c:1145
#61 0x0000000000742842 in PostgresMain (argc=<optimized out>, argv=<optimized out>, dbname=<optimized out>, username=<optimized out>) at postgres.c:4182
#62 0x00000000004801dd in BackendRun (port=0x1ec3f70) at postmaster.c:4361
#63 BackendStartup (port=0x1ec3f70) at postmaster.c:4033
#64 ServerLoop () at postmaster.c:1706
#65 0x00000000006d5d49 in PostmasterMain (argc=argc@entry=3, argv=argv@entry=0x1e98280) at postmaster.c:1379
#66 0x000000000048103f in main (argc=3, argv=0x1e98280) at main.c:228
(

(gdb) bt f
#0  0x00007f819e227cb9 in __memcpy_ssse3_back () from /lib64/libc.so.6
No symbol table info available.
#1  0x0000000000824076 in memcpy (__len=18446744073709551612, __src=0x6c35818, __dest=<optimized out>) at /usr/include/bits/string3.h:51
No locals.
#2  varstrfastcmp_locale (x=113465364, y=113465636, ssup=<optimized out>) at varlena.c:2135
        arg1 = 0x6c35814
        arg2 = 0x6c35924
        arg1_match = false
        sss = 0x6c33d60
        a1p = 0x6c35818 ""
        a2p = 0x6c35928 ""
        len1 = -4
        len2 = -4
        result = <optimized out>
#3  0x000000000087c6f4 in ApplySortComparator (ssup=0x6c33c40, isNull2=<optimized out>, datum2=<optimized out>, isNull1=false, datum1=<optimized out>) at ../../../../src/include/utils/sortsupport.h:224
        compare = <optimized out>
        isNull2 = false
        isNull1 = false
        ssup = 0x6c33c40
        datum2 = <optimized out>
        compare = <optimized out>
#4  comparetup_heap (a=<optimized out>, b=<optimized out>, state=0x6c337f0) at tuplesort.c:3560
        sortKey = 0x6c33c40
        ltup = {t_len = 248, t_self = {ip_blkid = {bi_hi = 0, bi_lo = 0}, ip_posid = 42884}, t_tableOid = 0, t_data = 0x6c357f8}
        rtup = {t_len = 248, t_self = {ip_blkid = {bi_hi = 0, bi_lo = 0}, ip_posid = 1}, t_tableOid = 0, t_data = 0x6c35908}
        tupDesc = 0x5ede208
        nkey = 1
        attno = 2
        datum1 = <optimized out>
        datum2 = <optimized out>
        isnull1 = false
        isnull2 = false
#5  0x00000000008793f5 in qsort_tuple (a=0x8311958, n=5272, cmp_tuple=0x87c2d0 <comparetup_heap>, state=state@entry=0x6c337f0) at qsort_tuple.c:112
        pa = <optimized out>
        pb = <optimized out>
        pc = <optimized out>
        pd = <optimized out>
        pl = <optimized out>
        pm = 0x8311970
        pn = <optimized out>
        d1 = <optimized out>
        d2 = <optimized out>
        r = <optimized out>
        presorted = 1
#6  0x000000000087d2db in tuplesort_sort_memtuples (state=state@entry=0x6c337f0) at tuplesort.c:3320
No locals.
#7  0x000000000087ee54 in tuplesort_performsort (state=state@entry=0x6c337f0) at tuplesort.c:1811
        oldcontext = 0x3832860
        __func__ = "tuplesort_performsort"
#8  0x0000000000628ce3 in ExecSort (pstate=0x5e50ed8) at nodeSort.c:118
        plannode = <optimized out>
        outerNode = 0x5e50fe8
        tupDesc = <optimized out>
        node = 0x5e50ed8
        estate = 0x3832970
        dir = ForwardScanDirection
        tuplesortstate = 0x6c337f0
        slot = <optimized out>
#9  0x0000000000609068 in ExecProcNodeInstr (node=0x5e50ed8) at execProcnode.c:461
        result = <optimized out>
#10 0x0000000000610d49 in ExecProcNode (node=0x5e50ed8) at ../../../src/include/executor/executor.h:237
No locals.
#11 fetch_input_tuple (aggstate=aggstate@entry=0x5e50aa0) at nodeAgg.c:406
        slot = <optimized out>
#12 0x0000000000612780 in agg_retrieve_direct (aggstate=0x5e50aa0) at nodeAgg.c:1736
        econtext = 0x5e50e28
        firstSlot = 0x674fa50
        numGroupingSets = 1
        node = 0x5c03788
        tmpcontext = 0x5e50cc8
        peragg = 0x6754918
        outerslot = <optimized out>
        nextSetSize = <optimized out>
        pergroups = 0x6755928
        result = <optimized out>
        hasGroupingSets = false
        currentSet = <optimized out>
        numReset = 1
        i = <optimized out>
#13 ExecAgg (pstate=0x5e50aa0) at nodeAgg.c:1551
        node = 0x5e50aa0
        result = 0x0
#14 0x0000000000609068 in ExecProcNodeInstr (node=0x5e50aa0) at execProcnode.c:461
        result = <optimized out>
#15 0x000000000060a765 in ExecScanFetch (recheckMtd=0x62be50 <SubqueryRecheck>, accessMtd=0x62be70 <SubqueryNext>, node=0x5e508e0) at execScan.c:95
        estate = 0x3832970
#16 ExecScan (node=0x5e508e0, accessMtd=0x62be70 <SubqueryNext>, recheckMtd=0x62be50 <SubqueryRecheck>) at execScan.c:162
        econtext = 0x5e509f0
        qual = 0x0
        projInfo = 0x675dbb8
#17 0x0000000000609068 in ExecProcNodeInstr (node=0x5e508e0) at execProcnode.c:461
        result = <optimized out>
#18 0x000000000061b365 in ExecProcNode (node=0x5e508e0) at ../../../src/include/executor/executor.h:237
No locals.
#19 MultiExecPrivateHash (node=0x5e50720) at nodeHash.c:164
        outerNode = <optimized out>
        hashkeys = 0x67662f0
        econtext = 0x5e50830
        hashvalue = 0
        hashtable = 0x6a7d468
#20 MultiExecHash (node=node@entry=0x5e50720) at nodeHash.c:114
No locals.
#21 0x0000000000609610 in MultiExecProcNode (node=node@entry=0x5e50720) at execProcnode.c:501
        __func__ = "MultiExecProcNode"
#22 0x000000000061bd98 in ExecHashJoinImpl (parallel=false, pstate=0x5e4f620) at nodeHashjoin.c:290
        outerNode = 0x5e4f8e0
        hashNode = <optimized out>
        econtext = <optimized out>
        node = 0x5e4f620
        joinqual = <optimized out>
        otherqual = <optimized out>
        hashtable = 0x6a7d468
        hashvalue = 0
        batchno = 48
        parallel_state = <optimized out>
#23 ExecHashJoin (pstate=0x5e4f620) at nodeHashjoin.c:565
No locals.
#24 0x0000000000609068 in ExecProcNodeInstr (node=0x5e4f620) at execProcnode.c:461
        result = <optimized out>
#25 0x000000000061b365 in ExecProcNode (node=0x5e4f620) at ../../../src/include/executor/executor.h:237
No locals.
#26 MultiExecPrivateHash (node=0x5e4f460) at nodeHash.c:164
        outerNode = <optimized out>
        hashkeys = 0x676ef80
        econtext = 0x5e4f570
        hashvalue = 0
        hashtable = 0x6a7cf48
#27 MultiExecHash (node=node@entry=0x5e4f460) at nodeHash.c:114
No locals.
#28 0x0000000000609610 in MultiExecProcNode (node=node@entry=0x5e4f460) at execProcnode.c:501
        __func__ = "MultiExecProcNode"
#29 0x000000000061bd98 in ExecHashJoinImpl (parallel=false, pstate=0x38344f8) at nodeHashjoin.c:290
        outerNode = 0x5c4b4b8
        hashNode = <optimized out>
        econtext = <optimized out>
        node = 0x38344f8
        joinqual = <optimized out>
        otherqual = <optimized out>
        hashtable = 0x6a7cf48
        hashvalue = 0
        batchno = 0
        parallel_state = <optimized out>
#30 ExecHashJoin (pstate=0x38344f8) at nodeHashjoin.c:565
No locals.
#31 0x0000000000609068 in ExecProcNodeInstr (node=0x38344f8) at execProcnode.c:461
        result = <optimized out>
#32 0x000000000061b365 in ExecProcNode (node=0x38344f8) at ../../../src/include/executor/executor.h:237
No locals.
#33 MultiExecPrivateHash (node=0x38343c8) at nodeHash.c:164
        outerNode = <optimized out>
        hashkeys = 0x6778c40
        econtext = 0x3834048
        hashvalue = 0
        hashtable = 0x6a7ce38
#34 MultiExecHash (node=node@entry=0x38343c8) at nodeHash.c:114
No locals.
#35 0x0000000000609610 in MultiExecProcNode (node=node@entry=0x38343c8) at execProcnode.c:501
        __func__ = "MultiExecProcNode"
#36 0x000000000061bd98 in ExecHashJoinImpl (parallel=false, pstate=0x3833148) at nodeHashjoin.c:290
        outerNode = 0x3833408
        hashNode = <optimized out>
        econtext = <optimized out>
        node = 0x3833148
        joinqual = <optimized out>
        otherqual = <optimized out>
        hashtable = 0x6a7ce38
        hashvalue = 0
        batchno = 96457208
        parallel_state = <optimized out>
#37 ExecHashJoin (pstate=0x3833148) at nodeHashjoin.c:565
No locals.
#38 0x0000000000609068 in ExecProcNodeInstr (node=0x3833148) at execProcnode.c:461
        result = <optimized out>
#39 0x0000000000628cd6 in ExecProcNode (node=0x3833148) at ../../../src/include/executor/executor.h:237
No locals.
#40 ExecSort (pstate=0x3832bd0) at nodeSort.c:107
        plannode = <optimized out>
        outerNode = 0x3833148
        tupDesc = <optimized out>
        node = 0x3832bd0
        estate = 0x3832970
        dir = ForwardScanDirection
        tuplesortstate = 0x6c0f6f0
        slot = <optimized out>
#41 0x0000000000609068 in ExecProcNodeInstr (node=0x3832bd0) at execProcnode.c:461
        result = <optimized out>
#42 0x0000000000610d49 in ExecProcNode (node=0x3832bd0) at ../../../src/include/executor/executor.h:237
No locals.
#43 fetch_input_tuple (aggstate=aggstate@entry=0x3832e30) at nodeAgg.c:406
        slot = <optimized out>
#44 0x0000000000612780 in agg_retrieve_direct (aggstate=0x3832e30) at nodeAgg.c:1736
        econtext = 0x3833098
        firstSlot = 0x677a550
        numGroupingSets = 1
        node = 0x5c06950
        tmpcontext = 0x3832ce0
        peragg = 0x67873d0
        outerslot = <optimized out>
        nextSetSize = <optimized out>
        pergroups = 0x6a752e8
        result = <optimized out>
        hasGroupingSets = false
        currentSet = <optimized out>
        numReset = 1
        i = <optimized out>
#45 ExecAgg (pstate=0x3832e30) at nodeAgg.c:1551
        node = 0x3832e30
        result = 0x0
#46 0x0000000000609068 in ExecProcNodeInstr (node=0x3832e30) at execProcnode.c:461
        result = <optimized out>
#47 0x00000000006025ba in ExecProcNode (node=0x3832e30) at ../../../src/include/executor/executor.h:237
No locals.
#48 ExecutePlan (execute_once=<optimized out>, dest=0xced3a0 <donothingDR>, direction=<optimized out>, numberTuples=0, sendTuples=true, operation=CMD_SELECT, use_parallel_mode=<optimized out>, planstate=0x3832e30,
    estate=0x3832970) at execMain.c:1723
        slot = <optimized out>
        current_tuple_count = 0
#49 standard_ExecutorRun (queryDesc=0x5c3ec68, direction=<optimized out>, count=0, execute_once=<optimized out>) at execMain.c:364
        estate = 0x3832970
        operation = CMD_SELECT
        dest = 0xced3a0 <donothingDR>
        sendTuples = <optimized out>
        oldcontext = 0x2078b00
        __func__ = "standard_ExecutorRun"
#50 0x00007f8191ade08d in pgss_ExecutorRun (queryDesc=0x5c3ec68, direction=ForwardScanDirection, count=0, execute_once=<optimized out>) at pg_stat_statements.c:892
        save_exception_stack = 0x7ffc304d0890
        save_context_stack = 0x0
        local_sigjmp_buf = {{__jmpbuf = {34056952, -4317634825662262871, 140721118841376, 96726120, 0, 0, 4318053775260225961, 4258026755233780137}, __mask_was_saved = 0, __saved_mask = {__val = {33258448, 2, 8603255,
                33258448, 5392503, 16, 0, 72057594037927936, 336, 1, 58927200, 58927472, 8861374, 7, 1, 58928688}}}}
#51 0x00007f80eb8bf5e1 in explain_ExecutorRun (queryDesc=0x5c3ec68, direction=ForwardScanDirection, count=0, execute_once=<optimized out>) at auto_explain.c:268
        save_exception_stack = 0x7ffc304d0cf0
        save_context_stack = 0x0
        local_sigjmp_buf = {{__jmpbuf = {34056952, -4317634825662262871, 140721118841376, 96726120, 0, 0, 4318053775153271209, 4257441557765001641}, __mask_was_saved = 0, __saved_mask = {__val = {336, 1, 58927200, 96726120,
                8861374, 2147483647, 1, 34048768, 6356030, 96726120, 96726120, 13907488, 140191684360137, 13554592, 810355232, 34056952}}}}
#52 0x00000000005a497c in ExplainOnePlan (plannedstmt=plannedstmt@entry=0x5c32cd8, into=into@entry=0x0, es=es@entry=0x207aaf8,
    queryString=queryString@entry=0x1ec5d58 "explain(analyze)\nSELECT site_office as site_gran, site_location AS bs, sect_mscid || ' (' || gsm_site_name_to_sect_name (site_name, sect_num, sect_name) || ')' AS sitename, data_u5
        dir = <optimized out>
        dest = <optimized out>
        queryDesc = 0x5c3ec68
        starttime = {tv_sec = 2774013, tv_nsec = 519809227}
        totaltime = 0
        eflags = <optimized out>
        instrument_option = <optimized out>
#53 0x00000000005a4c87 in ExplainOneQuery (query=<optimized out>, cursorOptions=<optimized out>, into=0x0, es=0x207aaf8,
    queryString=0x1ec5d58 "explain(analyze)\nSELECT site_office as site_gran, site_location AS bs, sect_mscid || ' (' || gsm_site_name_to_sect_name (site_name, sect_num, sect_name) || ')' AS sitename, data_utrancell.period as1
        plan = 0x5c32cd8
        planstart = {tv_sec = 2774013, tv_nsec = 111256886}
        planduration = {tv_sec = 0, tv_nsec = 408551920}
        params = 0x0
        queryString = 0x1ec5d58 "explain(analyze)\nSELECT site_office as site_gran, site_location AS bs, sect_mscid || ' (' || gsm_site_name_to_sect_name (site_name, sect_num, sect_name) || ')' AS sitename, data_utrancell.per.
        es = 0x207aaf8
        into = 0x0
        queryEnv = 0x0
        cursorOptions = <optimized out>
        query = <optimized out>
#54 0x00000000005a5174 in ExplainQuery (pstate=pstate@entry=0x2078dc8, stmt=stmt@entry=0x1fe1df8,
    queryString=queryString@entry=0x1ec5d58 "explain(analyze)\nSELECT site_office as site_gran, site_location AS bs, sect_mscid || ' (' || gsm_site_name_to_sect_name (site_name, sect_num, sect_name) || ')' AS sitename, data_u4
        l = 0x20e15a8
        es = 0x207aaf8
        tstate = <optimized out>
        rewritten = <optimized out>
        lc = <optimized out>
        timing_set = <optimized out>
        summary_set = <optimized out>
        __func__ = "ExplainQuery"
#55 0x00000000007469dd in standard_ProcessUtility (pstmt=pstmt@entry=0x1fe1ea8,
    queryString=queryString@entry=0x1ec5d58 "explain(analyze)\nSELECT site_office as site_gran, site_location AS bs, sect_mscid || ' (' || gsm_site_name_to_sect_name (site_name, sect_num, sect_name) || ')' AS sitename, data_u5
        parsetree = 0x1fe1df8
        isTopLevel = true
        isAtomicContext = false
        pstate = 0x2078dc8
        __func__ = "standard_ProcessUtility"
#56 0x00007f8191ae0025 in pgss_ProcessUtility (pstmt=0x1fe1ea8,
    queryString=0x1ec5d58 "explain(analyze)\nSELECT site_office as site_gran, site_location AS bs, sect_mscid || ' (' || gsm_site_name_to_sect_name (site_name, sect_num, sect_name) || ')' AS sitename, data_utrancell.period as5
        save_exception_stack = 0x7ffc304d0f10
        save_context_stack = 0x0
        local_sigjmp_buf = {{__jmpbuf = {32582624, -4317636606361366103, 34049336, 32848032, 32582624, 2, 4318053775111328169, 4258026755403256233}, __mask_was_saved = 0, __saved_mask = {__val = {0, 0, 8860127,
                140721118842400, 16, 32088528, 32847712, 24, 34049336, 140721118842400, 8860127, 32848032, 32582624, 32848032, 8926208, 2}}}}
        duration = {tv_sec = 33430760, tv_nsec = 33985024}
        bufusage = {shared_blks_hit = -140721118842111, shared_blks_read = 1028, shared_blks_dirtied = 257, shared_blks_written = 32, local_blks_hit = 528280977713, local_blks_read = 140721118842112, local_blks_dirtied = 4,
          local_blks_written = 0, temp_blks_read = 511101108338, temp_blks_written = 0, blk_read_time = {tv_sec = 140721118842111, tv_nsec = 140721118842160}, blk_write_time = {tv_sec = 2, tv_nsec = 0}}
        start = {tv_sec = 2774013, tv_nsec = 108551544}
        rows = <optimized out>
        bufusage_start = {shared_blks_hit = 900, shared_blks_read = 0, shared_blks_dirtied = 0, shared_blks_written = 0, local_blks_hit = <optimized out>, local_blks_read = <optimized out>,
          local_blks_dirtied = <optimized out>, local_blks_written = <optimized out>, temp_blks_read = <optimized out>, temp_blks_written = <optimized out>, blk_read_time = {tv_sec = <optimized out>,
            tv_nsec = <optimized out>}, blk_write_time = {tv_sec = <optimized out>, tv_nsec = <optimized out>}}
        parsetree = <optimized out>
#57 0x0000000000743e86 in PortalRunUtility (portal=0x1f12be0, pstmt=0x1fe1ea8, isTopLevel=<optimized out>, setHoldSnapshot=<optimized out>, dest=0x2078d38, completionTag=0x7ffc304d0e20 "") at pquery.c:1178
        utilityStmt = <optimized out>
        snapshot = 0x1f538a0
#58 0x0000000000744bc2 in FillPortalStore (portal=portal@entry=0x1f12be0, isTopLevel=isTopLevel@entry=true) at pquery.c:1038
        treceiver = 0x2078d38
        completionTag = "\000\377\377\377\000\000\000\000\260z\372\001\000\000\000\000\360\240\351\001\000\000\000\000\245c\207\000\000\000\000\000\020\214\a\002\000\000\000\000\260z\372\001\000\000\000\000\a\000\000\000\000\"
        __func__ = "FillPortalStore"
#59 0x000000000074564f in PortalRun (portal=<optimized out>, count=9223372036854775807, isTopLevel=<optimized out>, run_once=<optimized out>, dest=0x7f81a09e2dc8, altdest=0x7f81a09e2dc8, completionTag=0x7ffc304d1040 "")
    at pquery.c:768
        save_exception_stack = 0x7ffc304d1220
        save_context_stack = 0x0
        local_sigjmp_buf = {{__jmpbuf = {0, -4317636606344588887, 33431128, 140721118842944, 32582624, 2, 4318053774943556009, -4317633223726624343}, __mask_was_saved = 0, __saved_mask = {__val = {480, 10132060, 8746404, 1,
                32590832, 10476062, 34048768, 2, 1, 2, 112, 33431080, 32101840, 10233096, 8861374, 2}}}}
        result = <optimized out>
        nprocessed = <optimized out>
        saveTopTransactionResourceOwner = 0x1ed5a08
        saveTopTransactionContext = 0x1f53760
        saveActivePortal = 0x0
        saveResourceOwner = 0x1ed5a08
        savePortalContext = 0x0
        saveMemoryContext = 0x1f53760
        __func__ = "PortalRun"
#60 0x0000000000741577 in exec_simple_query (query_string=<optimized out>) at postgres.c:1145
        dest = <optimized out>
        oldcontext = <optimized out>
        parsetree_list = <optimized out>
        parsetree_item = <optimized out>
        save_log_statement_stats = <optimized out>
        was_logged = <optimized out>
        use_implicit_block = <optimized out>
        msec_str = "\000 \000\000\000\000\000\000?\252c\000\000\000\000\000\247\235\354\001\000\000\000\000p?\354\001\002\000\000"
        __func__ = "exec_simple_query"
#61 0x0000000000742842 in PostgresMain (argc=<optimized out>, argv=<optimized out>, dbname=<optimized out>, username=<optimized out>) at postgres.c:4182
        firstchar = 32267608
        input_message = {
          data = 0x1ec5d58 "explain(analyze)\nSELECT site_office as site_gran, site_location AS bs, sect_mscid || ' (' || gsm_site_name_to_sect_name (site_name, sect_num, sect_name) || ')' AS sitename, data_utrancell.period a}
        local_sigjmp_buf = {{__jmpbuf = {32292536, -4317633050175762007, 32259952, 0, 0, 32260608, 4318053775948091817, -4317633221742456407}, __mask_was_saved = 1, __saved_mask = {__val = {0, 32260608, 140194680307352, 0,
                206158430256, 140721118843760, 140721118843552, 32102304, 32080672, 16, 66, 32102304, 32292536, 0, 32292616, 32292784}}}}
        send_ready_for_query = false
        disable_idle_in_transaction_timeout = <optimized out>
        __func__ = "PostgresMain"
#62 0x00000000004801dd in BackendRun (port=0x1ec3f70) at postmaster.c:4361
        ac = 1
        secs = 595637552
        usecs = 441639
        i = 1
        av = 0x1ecbfe8
        maxac = <optimized out>
#63 BackendStartup (port=0x1ec3f70) at postmaster.c:4033
        bn = 0x1ec4200
        pid = 0
#64 ServerLoop () at postmaster.c:1706
        rmask = {fds_bits = {32, 0 <repeats 15 times>}}
        selres = <optimized out>
        now = <optimized out>
        readmask = {fds_bits = {120, 0 <repeats 15 times>}}
        last_lockfile_recheck_time = <optimized out>
        last_touch_time = 1542322298
        __func__ = "ServerLoop"
#65 0x00000000006d5d49 in PostmasterMain (argc=argc@entry=3, argv=argv@entry=0x1e98280) at postmaster.c:1379
        opt = <optimized out>
        status = <optimized out>
        userDoption = <optimized out>
        listen_addr_saved = true
        i = <optimized out>
        output_config_variable = <optimized out>
        __func__ = "PostmasterMain"
#66 0x000000000048103f in main (argc=3, argv=0x1e98280) at main.c:228
No locals.
(gdb)

> It'd also be interesting if toggling jit_inline_above_cost=-1,
> jit_optimize_above_cost=-1 and jit_tuple_deforming=false, each set
> individually, make a difference.

Crashes with jit_tuple_deforming=true (others off).

Doesn't crash with other combinations:
   Options: Inlining false, Optimization true, Expressions true, Deforming false
   Options: Inlining true, Optimization false, Expressions true, Deforming false

> Any way for me to reproduce this locally? I.e. can you narrow this down
> to a reproducible test case?

Will work on it.

Justin


Commits

  1. Fix jit compilation bug on wide tables.

  2. Verify that expected slot types match returned slot types.

  3. Rejigger materializing and fetching a HeapTuple from a slot.