1、DocValues说明
DocValues其实是Lucene在构建倒排索引时,会额外建立一个有序的正排索引(基于document=>field value的映射列表)。年龄、日期等非字符型的可以排序,就是因为建立了倒排索引,也建立了正排索引。
DocValues说白了就是正排索引,默认对字符串类型的不起作用,即:默认对不分词的字段是开启的,对分词字段无效(需要把fielddata设置为true才可以)。
DocValues存储在磁盘上,节省内存,对排序,分组和一些聚合操作能够大大提升性能。
DocValues映射表如下:json文档字符串在磁盘上对应的是文档doc1和文档doc2,每一个字段都会列出
{"birthday":"1985-11-11",age:23}
{"birthday":"1989-11-11",age:29}
-----------------------------------------------------------
document age birthday
doc1 23 1985-11-11
doc2 29 1989-11-11
2、演示关闭age字段DocValues,执行排序搜索会报错
准备数据:
PUT /lib { "settings":{ "number_of_shards":3, "number_of_replicas":0 }, "mappings":{ "user":{ "properties":{ "name":{"type":"text"}, "address":{"type":"text"}, "age":{"type":"integer"}, "interests":{"type":"text"}, "birthday":{"type":"date"} } } } }
put /lib/user/1 { "name":"zhaoliu", "address":"hei long jiang sheng tie ling shi", "age":50, "birthday":"1970-12-12", "interests":"xi huang hejiu,duanlian,lvyou" } put /lib/user/2 { "name":"zhaoming", "address":"bei jing hai dian qu qing he zhen", "age":20, "birthday":"1998-10-12", "interests":"xi huan hejiu,duanlian,changge" } put /lib/user/3 { "name":"lisi", "address":"bei jing hai dian qu qing he zhen", "age":23, "birthday":"1998-10-12", "interests":"xi huan hejiu,duanlian,changge" } put /lib/user/4 { "name":"wangwu", "address":"bei jing hai dian qu qing he zhen", "age":26, "birthday":"1998-10-12", "interests":"xi huan biancheng,tingyinyue,lvyou" } put /lib/user/5 { "name":"zhangsan", "address":"bei jing chao yang qu", "age":29, "birthday":"1988-10-12", "interests":"xi huan tingyinyue,changge,tiaowu" }
现在没有关闭age字段DocValues,执行排序搜索正常
GET lib/user/_search { "query": { "match_all": {} }, "sort": [ { "age": { "order": "desc" } } ] }
查询结果:
{ "took": 9, "timed_out": false, "_shards": { "total": 3, "successful": 3, "skipped": 0, "failed": 0 }, "hits": { "total": 5, "max_score": null, "hits": [ { "_index": "lib", "_type": "user", "_id": "1", "_score": null, "_source": { "name": "zhaoliu", "address": "hei long jiang sheng tie ling shi", "age": 50, "birthday": "1970-12-12", "interests": "xi huang hejiu,duanlian,lvyou" }, "sort": [ 50 ] }, { "_index": "lib", "_type": "user", "_id": "5", "_score": null, "_source": { "name": "zhangsan", "address": "bei jing chao yang qu", "age": 29, "birthday": "1988-10-12", "interests": "xi huan tingyinyue,changge,tiaowu" }, "sort": [ 29 ] }, { "_index": "lib", "_type": "user", "_id": "4", "_score": null, "_source": { "name": "wangwu", "address": "bei jing hai dian qu qing he zhen", "age": 26, "birthday": "1998-10-12", "interests": "xi huan biancheng,tingyinyue,lvyou" }, "sort": [ 26 ] }, { "_index": "lib", "_type": "user", "_id": "3", "_score": null, "_source": { "name": "lisi", "address": "bei jing hai dian qu qing he zhen", "age": 23, "birthday": "1998-10-12", "interests": "xi huan hejiu,duanlian,changge" }, "sort": [ 23 ] }, { "_index": "lib", "_type": "user", "_id": "2", "_score": null, "_source": { "name": "zhaoming", "address": "bei jing hai dian qu qing he zhen", "age": 20, "birthday": "1998-10-12", "interests": "xi huan hejiu,duanlian,changge" }, "sort": [ 20 ] } ] } }
现在重新建立索引,并添加上面的数据、执行排序搜索报错
PUT /lib { "settings":{ "number_of_shards":3, "number_of_replicas":0 }, "mappings":{ "user":{ "properties":{ "name":{"type":"text"}, "address":{"type":"text"}, "age":{ "type":"integer", "doc_values":false }, "interests":{"type":"text"}, "birthday":{"type":"date"} } } } }
执行查询
GET lib/user/_search { "query": { "match_all": {} }, "sort": [ { "age": { "order": "desc" } } ] }
查询结果,报错
{ "error": { "root_cause": [ { "type": "illegal_argument_exception", "reason": "Can't load fielddata on [age] because fielddata is unsupported on fields of type [integer]. Use doc values instead." } ], "type": "search_phase_execution_exception", "reason": "all shards failed", "phase": "query", "grouped": true, "failed_shards": [ { "shard": 0, "index": "lib", "node": "AJ3x6yc8TfKj6_zx6VRm0g", "reason": { "type": "illegal_argument_exception", "reason": "Can't load fielddata on [age] because fielddata is unsupported on fields of type [integer]. Use doc values instead." } } ] }, "status": 400 }
注意:
假如把age的doc values关闭,执行排序搜索会报错