Re: Crash report for some ICU-52 (debian8) COLLATE and work_mem values

	Peter Geoghegan wrote:

> I'd be interested to see if anything changed when -1 was passed as
> both sourceLength and targetLength to ucol_strcollUTF8(). You'd have
> to build Postgres yourself to test this, but it would just work, since
> we don't actually avoid NUL termination, even though in principled we
> could with ICU.

It doesn't seem to change the outcome:

*** varlena.c~	2017-07-10 22:26:20.000000000 +0200
--- varlena.c	2017-08-02 12:57:25.997265001 +0200
***************
*** 2137,2144 ****

				status = U_ZERO_ERROR;
				result =
ucol_strcollUTF8(sss->locale->info.icu.ucol,
!									     
    a1p, len1,
!									     
    a2p, len2,
									     
    &status);
				if (U_FAILURE(status))
					ereport(ERROR,
--- 2137,2144 ----

				status = U_ZERO_ERROR;
				result =
ucol_strcollUTF8(sss->locale->info.icu.ucol,
!									     
    a1p, -1, /* len1 */
!									     
    a2p, -1, /*len2,*/
									     
    &status);
				if (U_FAILURE(status))
					ereport(ERROR,


For the case when its ends up crashing in memcpy, the backtrace now looks
like this:

#0  __memcpy_sse2_unaligned ()
    at ../sysdeps/x86_64/multiarch/memcpy-sse2-unaligned.S:35
#1  0x00007f0156812e6b in memcpy (__len=8589934592, __src=0x7efbeac89010,
    __dest=<optimized out>) at
/usr/include/x86_64-linux-gnu/bits/string3.h:51
#2  ucol_CEBuf_Expand (ci=<optimized out>, status=0x7ffd49c4ca98,
    b=0x7ffd49c4ab60) at ucol.cpp:7009
#3  UCOL_CEBUF_PUT (status=0x7ffd49c4ca98, ci=0x7ffd49c4bde0, ce=1493173509,
    b=0x7ffd49c4ab60) at ucol.cpp:7022
#4  ucol_strcollRegular (sColl=sColl@entry=0x7ffd49c4bde0,
    tColl=tColl@entry=0x7ffd49c4bf90, status=status@entry=0x7ffd49c4ca98)
    at ucol.cpp:7163
#5  0x00007f0156818177 in ucol_strcollRegularUTF8 (coll=0x2286100,
    source=source@entry=0x3651829 "쳭喩zz",
    sourceLength=sourceLength@entry=-1, target=<optimized out>,
    targetLength=targetLength@entry=-1, status=status@entry=0x7ffd49c4ca98)
    at ucol.cpp:8023
#6  0x00007f0156819d36 in ucol_strcollUseLatin1UTF8 (status=<optimized out>,
    tLen=<optimized out>, target=<optimized out>, sLen=<optimized out>,
    source=<optimized out>, coll=<optimized out>) at ucol.cpp:8108
#7  ucol_strcollUTF8_52 (coll=<optimized out>,
    source=source@entry=0x3651829 "쳭喩zz", sourceLength=<optimized out>,
    sourceLength@entry=-1, target=<optimized out>,
    target@entry=0x3651799 "콗喩zx㎍", targetLength=targetLength@entry=-1, 
    status=status@entry=0x7ffd49c4ca98) at ucol.cpp:8770
#8  0x00000000007cc7df in varstrfastcmp_locale (x=56956968, y=56956824, 
    ssup=<optimized out>) at varlena.c:2139
#9  0x0000000000817168 in ApplySortAbbrevFullComparator (ssup=0x2280338, 
    isNull2=<optimized out>, datum2=<optimized out>, isNull1=<optimized out>, 
    datum1=<optimized out>) at
../../../../src/include/utils/sortsupport.h:263
#10 comparetup_datum (a=0x7f0141c14228, b=0x7f0141c14240, 
    state=<optimized out>) at tuplesort.c:4350
#11 0x0000000000815b1f in qsort_tuple (a=0x7f0141c14210, n=<optimized out>, 
    n@entry=106, cmp_tuple=cmp_tuple@entry=0x817060 <comparetup_datum>, 
    state=state@entry=0x2280128) at qsort_tuple.c:104
.....

Anyway I'll try to build ICU-52 without optimization to avoid these
<optimized out> parameters.

Also running the same tests with a self-compiled ICU-59 versus debian's
packaged
ICU-52 results in *no crash at all*. So maybe the root cause of these crashes
is 
that a critical fix in ICU has not been backported?

I note that the list of ICU locales available in pg_collation is different
though.
When compiled with ICU-59, initdb produces "only" 581 collname
matching '%icu%', versus 1741 with ICU-52. The ICU version
is the only thing that differs between both cases.
But anyway in the list of collations that crash for me with ICU-52, a fair
number
(~50) are still there and work in ICU-59, so I think the comparison is still
meaningful.


Best regards,
-- 
Daniel Vérité
PostgreSQL-powered mailer: http://www.manitou-mail.org
Twitter: @DanielVerite