Elasticsearch-98-Java API 常用的几种API及搜索模板调用

发表于 | 分类于

案例背景

以汽车零售为案例背景,简单来说,会涉及到三个数据,汽车信息,汽车销售记录,汽车4S店信息

准备工作

创建索引

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
PUT /car_shop
{
    "mappings": {
        "cars": {
            "properties": {
                "brand": {
                    "type": "text",
                    "analyzer": "ik_max_word",
                    "fields": {
                        "raw": {
                            "type": "keyword"
                        }
                    }
                },
                "name": {
                    "type": "text",
                    "analyzer": "ik_max_word",
                    "fields": {
                        "raw": {
                            "type": "keyword"
                        }
                    }
                }
            }
        }
    }
}

常用java Api使用

upsert Api

需求: 首先更新宝马320的价格为310000,如果这个汽车不存在的话就新增,如果存在的话,就做更新

注入client

1
2
@Autowired
private TransportClient client;

upsert操作

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
@Test
public void upsert() throws IOException, ExecutionException, InterruptedException {
    // index操作
    IndexRequest indexRequest = new IndexRequest("car_shop", "cars", "1")
            .source(jsonBuilder()
                .startObject()
                    .field("brand", "宝马")
                    .field("name","宝马320")
                    .field("price",320000)
                    .field("produce_date", "2018-01-01")
                .endObject()
            );

    // update操作
    UpdateRequest updateRequest = new UpdateRequest("car_shop", "cars", "1")
            .doc(jsonBuilder()
                .startObject()
                    .field("price", 310000)
                .endObject()
            ).upsert(indexRequest);

    // 客户端执行
    client.update(updateRequest).get();
}

第一次执行,因为我们刚创建的索引,并没有这条数据,所以第一次执行完成后,这条数据被添加了进去,price还是320000
第二次执行,数据已经存在了,就是走下面的updateRequest,把价格更新为310000

mget Api

场景: 一般来说,我们都可以在一些汽车网站上,或者混合销售多个品牌的汽车4s店的内部,都可以在系统里调出来多个汽车的信息,放在网页上,进行对比

mget,批量查询,一次性将多个document的数据查询出来

先再添加一条数据供测试

1
2
3
4
5
6
7
PUT /car_shop/cars/2
{
	"brand": "奔驰",
	"name": "奔驰C200",
	"price": 350000,
	"produce_date": "2017-01-05"
}

创建完成,用java的mget api,查询出来 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
@Test
public void mgetTest(){
    MultiGetResponse multiGetResponse = client.prepareMultiGet()
            .add("car_shop", "cars", "1")
            .add("car_shop", "cars", "2")
            .get();

    // 输出
    for (MultiGetItemResponse itemResponse : multiGetResponse) {
        GetResponse responses = itemResponse.getResponse();

        if (responses.isExists()){
            log.info("response:{}", responses.getSourceAsString());
        }
    }
}

控制台输出:

1
2
3
4
5
6
7
2019-01-23 11:07:00.302 [main] INFO  com.demo.elasticsearch.CarShopTests - response:{"brand":"宝马","name":"宝马320","price":310000,"produce_date":"2018-01-01"}
2019-01-23 11:07:00.303 [main] INFO  com.demo.elasticsearch.CarShopTests - response:{
	"brand": "奔驰",
	"name": "奔驰C200",
	"price": 350000,
	"produce_date": "2017-01-05"
}

两条数据就都查询出来了

bulk Api

场景: 有一个汽车销售公司,有很多家4s店,这些4S店的数据,都会在一段时间内陆续传递过来汽车的销售数据,现在希望能够在内存中缓存比如1000条销售数据,然后一次性批量上传到es中去

首先添加两条数据供后面使用

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
PUT /car_shop/sales/1
{
    "brand": "宝马",
    "name": "宝马320",
    "price": 320000,
    "produce_date": "2017-01-01",
    "sale_price": 300000,
    "sale_date": "2017-01-21"
}

PUT /car_shop/sales/2
{
    "brand": "宝马",
    "name": "宝马320",
    "price": 320000,
    "produce_date": "2017-01-01",
    "sale_price": 300000,
    "sale_date": "2017-01-21"
}

需求: 更新id是1的document中sale_price, 删除id是2的数据, 添加一条新的数据 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
@Test
public void bulkTest() throws IOException {
    BulkRequestBuilder bulkRequestBuilder = client.prepareBulk();

    // index操作,添加一条销售记录进去
    IndexRequestBuilder indexRequestBuilder = client.prepareIndex("car_shop", "sales", "3")
            .setSource(jsonBuilder()
                .startObject()
                    .field("brand", "奔驰")
                    .field("name", "奔驰C200")
                    .field("price", 350000)
                    .field("produce_date", "2017-01-05")
                    .field("sale_price", 340000)
                    .field("sale_date", "2017-02-03")
                .endObject()
            );

    // update操作 更新一条id是1的数据
    UpdateRequestBuilder updateRequestBuilder = client.prepareUpdate("car_shop", "sales", "1")
            .setDoc(jsonBuilder()
                .startObject()
                    .field("sale_price", 290000)
                .endObject()
            );

    // 删除操作 删除id是2的数据
    DeleteRequestBuilder deleteRequestBuilder = client.prepareDelete("car_shop", "sales", "2");

    // 请求都添加到bulk中
    bulkRequestBuilder.add(indexRequestBuilder)
            .add(updateRequestBuilder)
            .add(deleteRequestBuilder);

    // 发送请求
    BulkResponse responses = bulkRequestBuilder.get();

}

执行完成后再来看一下sales中的数据

1
GET /car_shop/sales/_search

返回值 :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 1,
    "hits": [
      {
        "_index": "car_shop",
        "_type": "sales",
        "_id": "1",
        "_score": 1,
        "_source": {
          "brand": "宝马",
          "name": "宝马320",
          "price": 320000,
          "produce_date": "2017-01-01",
          "sale_price": 290000,
          "sale_date": "2017-01-21"
        }
      },
      {
        "_index": "car_shop",
        "_type": "sales",
        "_id": "3",
        "_score": 1,
        "_source": {
          "brand": "奔驰",
          "name": "奔驰C200",
          "price": 350000,
          "produce_date": "2017-01-05",
          "sale_price": 340000,
          "sale_date": "2017-02-03"
        }
      }
    ]
  }
}

已经完成了, 再看一下代码里面, 就是用的工厂模式,先搞了一个bulk的builder, 然后将每个请求写出来,添加到工厂中去,最后发送请求,就ok了

scroll Api

对大数据的批量查询

先插入一条数据供测试用

1
2
3
4
5
6
7
8
9
PUT /car_shop/sales/4
{
    "brand": "宝马",
    "name": "宝马320",
    "price": 320000,
    "produce_date": "2017-01-01",
    "sale_price": 280000,
    "sale_date": "2017-01-25"
}

现在,宝马的销售记录是有两条数据的,然后就查询宝马的数据,分两次查询,每次一条,使用scroll Api 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
public void scrollTests(){

    // scroll查询 时间为60s,查询是宝马的数据每次查询一条
    SearchResponse response = client.prepareSearch("car_shop")
            .setTypes("sales")
            .setScroll(new Scroll(new TimeValue(60000)))
            .setQuery(QueryBuilders.termQuery("brand.keyword", "宝马"))
            .setSize(1)
            .get();

    // 继续往下查询
    while (response.getHits().getHits().length > 0){
        for (SearchHit hit : response.getHits().getHits()) {
            // 拿到每条数据去处理
            log.info("hit:{}", hit.getSourceAsString());
        }

        // 继续下一次查询
        response = client.prepareSearchScroll(response.getScrollId())
                .setScroll(new TimeValue(60000))
                .execute()
                .actionGet();
        
    }
}

控制台输出:

1
2
3
4
5
6
7
8
9
10
2019-01-23 14:38:35.598 [main] INFO  com.demo.elasticsearch.CarShopTests - hit:{
    "brand": "宝马",
    "name": "宝马320",
    "price": 320000,
    "produce_date": "2017-01-01",
    "sale_price": 280000,
    "sale_date": "2017-01-25"
}

2019-01-23 14:38:35.620 [main] INFO  com.demo.elasticsearch.CarShopTests - hit:{"brand":"宝马","name":"宝马320","price":320000,"produce_date":"2017-01-01","sale_price":290000,"sale_date":"2017-01-21"}

两条数据都查询了出来

调用搜索模板

首先,需要创建一个搜索模板,在es的/config/scripts目录下新建page_query_by_brand.mustache模板文件,内容如下: 

1
2
3
4
5
6
7
8
9
{
  "from": {{from}},
  "size": {{size}},
  "query": {
    "match": {
      "brand.keyword": "{{brand}}" 
    }
  }
}

然后,调用模板查询

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
public void searchTemplate(){
    // 请求参数
    Map<String,Object> map = new HashMap<>(3);
    map.put("from", 0);
    map.put("size", 1);
    map.put("brand", "宝马");

    SearchResponse response = new SearchTemplateRequestBuilder(client)
            .setScript("page_query_by_brand")
            .setScriptType(ScriptType.FILE)
            .setScriptParams(map)
            .setRequest(new SearchRequest("car_shop").types("sales"))
            .get()
            .getResponse();

    for (SearchHit hit : response.getHits().getHits()) {
        log.info("hit:{}", hit.getSourceAsString());
    }
}

控制台输出:

1
2
3
4
5
6
7
8
2019-01-23 15:07:57.875 [main] INFO  com.demo.elasticsearch.CarShopTests - hit:{
    "brand": "宝马",
    "name": "宝马320",
    "price": 320000,
    "produce_date": "2017-01-01",
    "sale_price": 280000,
    "sale_date": "2017-01-25"
}