ElasticSearch与Java集成-RestHighLevelClient

1.  简介

　　RestHighLevelClient是官方指定的Java连接ElasticSearch的API。

　　需要使用maven引用以下依赖：

<dependency>
　　<groupId>org.elasticsearch.client</groupId>
   <artifactId>elasticsearch-rest-high-level-client</artifactId>
   <version>6.5.2</version>
</dependency>

　　注意：以上的依赖版本可以根据你使用的ES的版本来定，向下兼容，但是无法向上兼容

2. 创建客户端

　　创建客户端在网上有很多种配置，下面是最简单的一种，需要其他中详情配置的可以自己查。

    static String ip = "localhost";
    static int port = 9200;
    static RestHighLevelClient restHighLevelClient = null;
    static TransportClient client = null;
    
    static  RestHighLevelClient initClient(){
        //这里的RestClient.builder(new HttpHost(ip,port),.........)支持多个httphost连接，也就是支持连接多个elasticsearch
        restHighLevelClient = new RestHighLevelClient(RestClient.builder(new HttpHost(ip,port)));
    }

    static Connection getInstance(){
        synchronized (ElasticSearchFactory.class){
            if (restHighLevelClient == null){
             restHighLevelClient  =   initClient();
            }
        }
        return connection;
    }

注意：以下操作都是个使用了RestHighLevelClient的API对ElasticSearch进行操作，各版本之间的语法可能略有区别。

3. 索引操作

3.1 创建索引　

　　创建索引操作，最简单的操作就是只传一个索引名称，CreateIndexRequest request = new CreateIndexRequest(index); 至于下面组织setting和mappings的XContentBuilder对象，是可以不传，也可以按照自己的业务自行定义。

@Autowired
private RestHighLevelClient client;

　　下面出现的变量client，都是引用的RestHighLevelClient

    /**
     * 创建索引
     * @param index  索引名称
     * @throws IOException e
     */
    public void createIndex(String index) throws IOException {
        CreateIndexRequest request = new CreateIndexRequest(index);
        XContentBuilder builder = JsonXContent.contentBuilder()
                .startObject()
                    .startObject("settings").field("number_of_shards", 3).field("number_of_replicas", 1).endObject()
                    .startObject("mappings")
                        .startObject("doc")
                            .startObject("properties")
                                .startObject("title").field("type", "text").endObject()
                                .startObject("content").field("type", "text").field("index", true).endObject()
                                .startObject("uniqueId").field("type", "keyword").field("index", false).endObject()
                                .startObject("created").field("type", "date").field("format", "strict_date_optional_time||epoch_millis").endObject()
                            .endObject()
                        .endObject()
                    .endObject()
                .endObject();
        request.source(builder);
        CreateIndexResponse createIndexResponse = client.indices().create(request, RequestOptions.DEFAULT);
        System.out.println("createIndex: " + JSONUtil.toJsonStr(createIndexResponse));
    }

　

3.2 查询索引列表

    /**
     * 查询索引列表
     * @return Set<String>
     * @throws IOException e
     */
    public Set<String> indices() throws IOException {
        GetAliasesRequest request = new GetAliasesRequest();
        GetAliasesResponse getAliasesResponse =  client.indices().getAlias(request,RequestOptions.DEFAULT);
        Map<String, Set<AliasMetaData>> aliases = getAliasesResponse.getAliases();
        return aliases.keySet();
    }

3.3 判断索引存在

    /**
     * 判断索引是否存在，在创建索引之前使用
     * @param index 索引名称
     * @return boolean
     * @throws IOException e
     */
    public boolean existsIndex(String index) throws IOException {
        GetIndexRequest request = new GetIndexRequest();
        request.indices(index);
        return client.indices().exists(request, RequestOptions.DEFAULT);
    }

4. 数据操作

4.1 新增数据

    /**
     * 新增数据
     * @param index 索引
     * @param type  索引类型
     * @param id    数据ID
     * @param object 数据对象
     */public void addData(String index, String type, String id, JSONObject object) throws IOException {
        CreateIndexRequest request = new CreateIndexRequest();
        client.indices().create(request, RequestOptions.DEFAULT);

        IndexRequest indexRequest = new IndexRequest(index, type, id);
        indexRequest.source(JSONUtil.toJsonStr(object), XContentType.JSON);
        IndexResponse indexResponse = client.index(indexRequest, RequestOptions.DEFAULT);
        log.info("新增结果: " + JSONUtil.toJsonStr(indexResponse));
    }

4.2 判断数据是否存在

    /**
     * 判断数据是否存在
     * @param index 索引
     * @param type  索引类型
     * @param id    数据ID
     * @return boolean
     */
    public boolean exists(String index, String type, String id) throws IOException {
        GetRequest getRequest = new GetRequest(index, type, id);
        getRequest.fetchSourceContext(new FetchSourceContext(false));
        getRequest.storedFields("_none_");
        boolean exists = client.exists(getRequest, RequestOptions.DEFAULT);
        log.info("判断数据是否存在：" + exists);
        return exists;
    }

4.3 修改数据

    /**
     * 修改数据
     * @param index  索引
     * @param type   索引类型
     * @param id     数据ID
     * @param object 修改的数据
     */
    public void updateData(String index, String type, String id, JSONObject object) throws IOException {
        UpdateRequest request = new UpdateRequest(index, type, id);
        request.doc(JSONUtil.toJsonStr(object), XContentType.JSON);
        UpdateResponse updateResponse = client.update(request, RequestOptions.DEFAULT);
        log.info("修改数据结果: " + JSONUtil.toJsonStr(updateResponse));
    }

4.4 删除数据

    /**
     * 删除数据
     * @param index 索引
     * @param type  索引类型
     * @param id    数据ID
     */
    public void deleteData(String index, String type, String id) throws IOException {
        DeleteRequest deleteRequest = new DeleteRequest(index, type, id);
        DeleteResponse response = client.delete(deleteRequest, RequestOptions.DEFAULT);
        System.out.println("删除结果: " + JSONUtil.toJsonStr(response));
    }

4.5 批量操作数据

　　下面的方法是模拟了批量新增、修改、删除，各位可以按照自己的需求修改。

public void bulk(List<UserBean> userList) throws IOException {
        // 批量增加
        BulkRequest bulkAddRequest = new BulkRequest();
        for (UserBean user : userList) {
            IndexRequest indexRequest = new IndexRequest(INDEX_TEST, TYPE_TEST, user.getId().toString());
            indexRequest.source(JSONUtil.toJsonStr(user), XContentType.JSON);
            bulkAddRequest.add(indexRequest);
        }
        
        BulkResponse bulkAddResponse = client.bulk(bulkAddRequest, RequestOptions.DEFAULT);
        System.out.println("bulkAdd: " + JSONUtil.toJsonStr(bulkAddResponse));

        // 批量更新
        BulkRequest bulkUpdateRequest = new BulkRequest();
        for (UserBean user : userList) {
            user.setName(user.getName() + " updated");
            UpdateRequest updateRequest = new UpdateRequest(INDEX_TEST, TYPE_TEST, user.getId().toString());
            updateRequest.doc(JSONUtil.toJsonStr(user), XContentType.JSON);
            bulkUpdateRequest.add(updateRequest);
        }
        BulkResponse bulkUpdateResponse = client.bulk(bulkUpdateRequest, RequestOptions.DEFAULT);
        System.out.println("bulkUpdate: " + JSONUtil.toJsonStr(bulkUpdateResponse));
        
        // 批量删除
        BulkRequest bulkDeleteRequest = new BulkRequest();
        for (UserBean user : userList) {
            DeleteRequest deleteRequest = new DeleteRequest(INDEX_TEST, TYPE_TEST, user.getId().toString());
            bulkDeleteRequest.add(deleteRequest);
        }
        BulkResponse bulkDeleteResponse = client.bulk(bulkDeleteRequest, RequestOptions.DEFAULT);
        System.out.println("bulkDelete: " + JSONUtil.toJsonStr(bulkDeleteResponse));
    }

5. 数据查询

5.1  关键字查询，匹配单个字段

　　查询条件都是通过QueryBuilders对象进行创建。

　　SearchSourceBuilder是用来拼接其他查询参数，比如分页参数，返回的字段值，排序等。

　　SearchRequest用来指定查询的索引index和type

　　SearchResponse 是用来接收查询结果，response.getHits().getHits()是查询的数据结果集。

    /**
     * 关键字查询，匹配单个字段
     * @param index  索引
     * @param type   索引类型
     * @param field  查询字段
     * @param value  查询参数
     * @return String
     */
    public String searchMatch(String index, String type, String field, String value) throws IOException {
        MatchQueryBuilder matchQueryBuilder = QueryBuilders.matchQuery(field, value);
        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        sourceBuilder.query(matchQueryBuilder);
        sourceBuilder.from(0);   // 查询起始行，分页参数
        sourceBuilder.size(100); // 获取记录数，默认10，分页参数
        sourceBuilder.fetchSource(new String[] { "id", "name" }, new String[] {}); // 第一个参数是要获取字段，第二个是要过滤的字段，默认获取返回全部字段
        sourceBuilder.sort("id", SortOrder.DESC); // 排序

        SearchRequest searchRequest = new SearchRequest(index).types(type).source(sourceBuilder);
        SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
        SearchHits hits = response.getHits();
        SearchHit[] searchHits = hits.getHits();
        for (SearchHit hit : searchHits) {
            log.info("查询结果：" + hit.getSourceAsString());
        }
        return JSONUtil.toJsonStr(searchHits);
    }

5.2 关键字查询，匹配多个字段

    /**
     * 关键字查询，匹配多个字段
     * @param index  索引
     * @param type   索引类型
     * @param fields  查询多个字段
     * @param value  查询参数
     * @return String
     */public String searchMultiMatch(String index, String type, List<String> fields, String value) throws IOException {
        MultiMatchQueryBuilder multiMatchQueryBuilder = QueryBuilders.multiMatchQuery(value, fields.toArray(new String[]{}));
        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        sourceBuilder.query(multiMatchQueryBuilder).from(0).size(100);
        SearchRequest searchRequest = new SearchRequest(index).types(type).source(sourceBuilder);
        SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
        ......
    }

5.3 关键字查询，范围查询

    /**
     * 关键字查询，范围查询（适用于数字类型、日期类型）
     * @param index  索引
     * @param type   索引类型
     * @param field  查询字段
     * @param start  开始参数
     * @param end    结束参数
     */
    @Override
    public String searchRange(String index, String type, String field, int start, int end) throws IOException {
        RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery(field).from(start).to(end);
        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        sourceBuilder.query(rangeQueryBuilder).from(0).size(100);
        SearchRequest searchRequest = new SearchRequest(index).types(type).source(sourceBuilder);
        SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
        ......
    }

5.4 关键字精确查询

    /**
     * 关键字精确查询（精确值可能是数字、时间、布尔或者not_analyzed的字符串）
     * @param index  索引
     * @param type   索引类型
     * @param field  查询字段
     * @param value  查询值
     */public String searchTerm(String index, String type, String field, String value) throws IOException {
        TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery(field, value);
        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        sourceBuilder.query(termQueryBuilder).from(0).size(100);
        SearchRequest searchRequest = new SearchRequest(index).types(type).source(sourceBuilder);
        SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
        ......
    }

5.5 组合查询

    /**
     * 组合查询
     * must 文档 必须 匹配这些条件才能被包含进来。
     * must_not 文档 必须不 匹配这些条件才能被包含进来。
     * should 如果满足这些语句中的任意语句，将增加 _score ，否则，无任何影响。它们主要用于修正每个文档的相关性得分。
     * filter 必须 匹配，但它以不评分、过滤模式来进行。这些语句对评分没有贡献，只是根据过滤标准来排除或包含文档。
     * @param index  索引
     * @param type   索引类型
     * @param field  查询字段
     * @param value  查询值
     */
    public String searchBool(String index, String type, String field, String value) throws IOException {
        BoolQueryBuilder builder = QueryBuilders.boolQuery();
        builder.must(QueryBuilders.matchQuery("title", "我是谁"));
        builder.mustNot(QueryBuilders.matchQuery("title", "你"));
        builder.should(QueryBuilders.matchQuery("desc", "啊啊啊"));
        builder.filter(QueryBuilders.termsQuery("author", "老王"));
        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        sourceBuilder.query(builder).from(0).size(100);
        SearchRequest searchRequest = new SearchRequest(index).types(type).source(sourceBuilder);
        SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
        ......
    }

6. 聚合

6.1 基础聚合

        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        TermsAggregationBuilder categoryAggregationBuilder = AggregationBuilders.terms("byCategory").field("category");
        TermsAggregationBuilder sourceAggregationBuilder = AggregationBuilders.terms("bySource").field("source");
        sourceBuilder.aggregation(categoryAggregationBuilder).aggregation(sourceAggregationBuilder);

        SearchRequest searchRequest = new SearchRequest(INDEX_TEST).types(TYPE_TEST).source(sourceBuilder);
        SearchResponse response = rhlClient.search(searchRequest, RequestOptions.DEFAULT);
        if (response != null) {
            ParsedLongTerms resourceTerms = response.getAggregations().get("byCategory");
            for (Terms.Bucket entry : resourceTerms.getBuckets()) {
                System.out.println("统计key=" + entry.getKeyAsString() + "， 统计数量："+  entry.getDocCount());
            }
        } else {
            log.error("资源状态聚合查询失败：" + searchRequest.toString());
        }

 　　代码解释：

　　1. 使用AggregationBuilders创建agg对象，terms的值是此次聚合的的别名，field是要聚合的字段

　　2. sourceBuilder.aggregation(****) 用来传入agg对象，可以同时接收多个agg对象，表示同时对多个维度进行统计。

　　3. response.getAggregations().get("byCategory"); 结果集中，getAggregations获取聚合对象，get("byCategory")获取byCategory别名的统计结果，结果集要的数据类型要区别清除

6.2 聚合过滤

　　在执行聚合操作之前，先用过滤条件把需要聚合的数据过滤出来。

　　注意：聚合、过滤可以嵌套，并且随意组合，造成返回的SearchResponse对象，组装的数据格式无法统一，所以只能按照自己的业务提取最终数据。

        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        sourceBuilder.aggregation(AggregationBuilders
                        .filter("categoryFilter", QueryBuilders.termQuery("status", 3))
                        .subAggregation(AggregationBuilders.terms("byCategory").field("category")))
                .aggregation(AggregationBuilders.terms("bySource").field("source"))
                .size(0);
        // 设置索引，类型
        SearchRequest searchRequest = new SearchRequest(properties.getEsIndex()).types(properties.getEsType()).source(sourceBuilder);
        SearchResponse response;
        try {
            response = rhlClient.search(searchRequest, RequestOptions.DEFAULT);
            if (response != null) {
                // 分享状态结果
                ParsedFilter filter = response.getAggregations().get("categoryFilter");
                ParsedStringTerms sharedTerms = filter.getAggregations().get("byCategory");
                for (Terms.Bucket entry : sharedTerms.getBuckets()) {
                    System.out.println("统计key=" + entry.getKeyAsString() + "， 统计数量："+  entry.getDocCount());
                }
                // 资源状态结果
                ParsedLongTerms resourceTerms = response.getAggregations().get("bySource");
                for (Terms.Bucket entry : resourceTerms.getBuckets()) {
                    System.out.println("统计key=" + entry.getKeyAsString() + "， 统计数量："+  entry.getDocCount());

                }
            }
        } catch (Exception e) {
            ......
        }

相同的主题的其他文章：

　　https://mp.weixin.qq.com/s/94j352V5xt9Tpqe00-eiQA