现在有两条document:
doc1:I really liked my small dogs, and I think my mom also liked them.
doc2:He never liked any dogs, so I hope that my mom will not expect me to liked him.
1.分词,初步的倒排索引的建立(*代表有,空代表无):
word doc1 doc2 I * * really * liked * * my * * small * dogs * and * think * mom * * also * them * He * never * any * so * hope * that * will * not * expect * me * to * him *
这边演示了一下倒排索引最简单的建立的一个过程
假设进行搜索:mother like little dog,结果是:不可能有任何结果
过程是将mother like little dog进行一个个拆分,然后进行匹配,无任何匹配信息。
这个是不是我们想要的搜索结果???绝对不是,因为在我们看来,mother和mom有区别吗?同义词,都是妈妈的意思。like和liked有区别吗?没有,都是喜欢的意思,只不过一个是现在时,一个是过去时。little和small有区别吗?同义词,都是小小的。dog和dogs有区别吗?狗,只不过一个是单数,一个是复数。
2.es在建立倒排索引的时候进行了normalization操作
normalization,建立倒排索引的时候,会执行一个操作,也就是说对拆分出的各个单词进行相应的处理,以提升后面搜索的时候能够搜索到相关联的文档的概率。
normalization的意思是进行时态的转换,单复数的转换,同义词的转换,大小写的转换。
mom —> mother liked —> like small —> little dogs —> dog
重新建立倒排索引,加入normalization,再次用mother liked little dog搜索,就可以搜索到了
word doc1 doc2 I * * really * like * * liked --> like my * * little * small --> little dog * * dogs --> dog and * think * mom * * also * them * He * never * any * so * hope * that * will * not * expect * me * to * him *
进行搜索:mother like little dog,结果:doc1和doc2都会搜索出来