Re: string_to_array eats too much memory?

>> tsvector size should not be greater than 1Mb however.
> 
> Is this documented somewhere? Also I noticed that tsearch2 treats ":"
> as a special character. Are there any special characters? If so where
> are they documented?
http://www.sai.msu.su/~megera/wiki/Tsearch_V2_in_Brief
Limitations

	13.1 2048 bytes for lexems
	13.2 ts_vector has limit about 1Mb. Exact value depends on
		quantity of position information. If there is no any position
                 information, then sum of length of lexem must be less than 1Mb,
                 otherwise, sum of length of and pos. info.
                 Positional information uses 2 bytes per each
		position and 2 bytes per lexem with pos info. The number of
                 lexems is limited by 4^32, so in practice it's unlimited.
	13.3 ts_query:
                 Number of entries (nodes, i.e sum of lexems and operation)
                 is limited: internal representation is in polish notation
                 and position of one operand is pointed by int2, so it's
                 rather soft limit.
                 In any case, low range of limit - 32768 nodes.
		Notice: ts_query doesn't designed for storing in table and
                 is optimized for speed, not for size.
	13.4 Positional information in ts_vector:
		13.4.1 Value of position may not be greater than 2^14 (16384),
                        any value greater than this limit will be replaced
                        by 16383.
		13.4.2 Only 256 positional info per lexem.


Some useful articles
http://www.sai.msu.su/~megera/postgres/gist/tsearch/V2/docs/HOWTO-parser-tsearch2.html
http://www.sai.msu.su/~megera/postgres/gist/tsearch/V2/docs/custom-dict.html


-- 
Teodor Sigaev                                   E-mail: teodor@sigaev.ru
                                                    WWW: http://www.sigaev.ru/