improve support of agglutinative languages (query with compound words).
Teodor Sigaev <teodor@sigaev.ru>
improve support of agglutinative languages (query with compound words).
regression=# select to_tsquery( '\'fotballklubber\'');
to_tsquery
------------------------------------------------
'fotball' & 'klubb' | 'fot' & 'ball' & 'klubb'
(1 row)
So, changed interface to dictionaries, lexize method of dictionary shoud return
pointer to aray of TSLexeme structs instead of char**. Last element should
have TSLexeme->lexeme == NULL.
typedef struct {
/* number of variant of split word , for example
Word 'fotballklubber' (norwegian) has two varian to split:
( fotball, klubb ) and ( fot, ball, klubb ). So, dictionary
should return:
nvariant lexeme
1 fotball
1 klubb
2 fot
2 ball
2 klubb
*/
uint16 nvariant;
/* currently unused */
uint16 flags;
/* C-string */
char *lexeme;
} TSLexeme;
Files
| Path | Change | +/− |
|---|---|---|
| contrib/tsearch2/dict.c | modified | +8 −8 |
| contrib/tsearch2/dict_ex.c | modified | +4 −4 |
| contrib/tsearch2/dict.h | modified | +23 −0 |
| contrib/tsearch2/dict_ispell.c | modified | +9 −10 |
| contrib/tsearch2/dict_snowball.c | modified | +3 −5 |
| contrib/tsearch2/dict_syn.c | modified | +4 −5 |
| contrib/tsearch2/gendict/dict_tmpl.c.IN | modified | +5 −5 |
| contrib/tsearch2/ispell/spell.c | modified | +39 −23 |
| contrib/tsearch2/ispell/spell.h | modified | +4 −3 |
| contrib/tsearch2/query.c | modified | +33 −10 |
| contrib/tsearch2/ts_cfg.c | modified | +13 −12 |
| contrib/tsearch2/ts_cfg.h | modified | +1 −0 |