Re: pg_trgm word_similarity inconsistencies or bug

Artur Zakirov <a.zakirov@postgrespro.ru>

From: Arthur Zakirov <a.zakirov@postgrespro.ru>
To: Cristiano Coelho <cristianocca@hotmail.com>
Cc: "pgsql-bugs@postgresql.org" <pgsql-bugs@postgresql.org>
Date: 2017-10-28T08:22:29Z
Lists: pgsql-bugs, pgsql-hackers
On Fri, Oct 27, 2017 at 06:48:08PM +0000, Cristiano Coelho wrote:
> Hello all, this is related to postgres 9.6 (9.6.4) and a good description can be found here https://stackoverflow.com/questions/46966360/postgres-word-similarity-not-comparing-words
> 
> But in summary, word_similarity doesn’t seem to do exactly what the docs say, since it will match trigrams from multiple words rather tan doing a word by word comparison.
> 
> Below is a table with output and expected output, thanks to kiln from stackoverflow to provide it.
> 

Interesting. An klin's answer from stackoverflow.com is right.

The initial example can be reduced to the next:

=# select word_similarity('sage', 'age sag');
 word_similarity 
-----------------
               1

It computes maximum similarity using closest trigrams not considering order of
'sage' trigrams. It determines that all
trigrams from 'sage' match trigrams from 'age sag'.

Initial order of 'age sag' trigrams:
'  a', ' ag', 'age', 'ge ', '  s', ' sa', 'sag', 'ag '
                ^                           ^
                |from                       |to
Sorted 'sage' trigrams (all of them occured within 'age sag' trigrams
continuously):
'  s', ' sa', 'age', 'ge ', 'sag'

Maybe the problem should be solved by considering 'sage' trigrams
initial order.

-- 
Arthur Zakirov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company


Commits

  1. Update trigram example in docs to correct state

  2. Add strict_word_similarity to pg_trgm module

  3. Rework word_similarity documentation, make it close to actual algorithm.