Re: string_to_array eats too much memory?

> > I'm testing how GIN scales.
> 
> Have a look at http://www.sigaev.ru/cvsweb/cvsweb.cgi/ftsbench/ - utility is 
> specially developed for measuring performance of full-text solutions ( now it 
> supports PgSQL( GiST, GIN ) and MySQL ). Right now I'm searching good query 
> statistic for simulate load, but this data is a closed information in 
> internet-wide search engines :(

Thanks.

> GIN itself is a just a tool for speedup searches, linguistic part is still in 
> tsearch2.
> 
> It's possible to use tsearch2 without any indexes at all. GiST and GIN is  a way 
> to speedup searches.
> 
> Of course, you can develop another framework for full text search and framework 
> may use GIN as it wish :)

Porblem with Japanese is, it's an agglutinative language and we need
to separate each word from a sentence. So, I need to modify tsearch2
anyway (I know someone from Japan is working on this).

BTW, can tsearch2 handle ~70k words in a document?
--
Tatsuo Ishii
SRA OSS, Inc. Japan