SprignBoot整合Spring Data Elasticsearch
Posted edda_huang
tags:
篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了SprignBoot整合Spring Data Elasticsearch相关的知识,希望对你有一定的参考价值。
一、原生java整合elasticsearch的API地址
https://www.elastic.co/guide/en/elasticsearch/client/java-api/6.2/java-docs.html
二、Spring Data的官网
http://projects.spring.io/spring-data/
Spring Data 是的使命是给各种数据访问提供统一的编程接口,不管是关系型数据库(如mysql),还是非关系数据库(如Redis),或者类似Elasticsearch这样的索引数据库。从而简化开发人员的代码,提高开发效率。
包含很多不同数据操作的模块:
Spring Data Elasticsearch的页面:https://projects.spring.io/spring-data-elasticsearch/
特征:
-
支持Spring的基于
@Configuration
的java配置方式,或者XML配置方式 -
提供了用于操作ES的便捷工具类
ElasticsearchTemplate
。包括实现文档到POJO之间的自动智能映射。 -
利用Spring的数据转换服务实现的功能丰富的对象映射
-
基于注解的元数据映射方式,而且可扩展以支持更多不同的数据格式
-
根据持久层接口自动生成对应实现方法,无需人工编写基本操作代码(类似mybatis,根据接口自动得到实现)。当然,也支持人工定制查询
三、SprignBoot整合Spring Data Elasticsearch
(1)创建项目
创建spring boot工程
(2)依赖
- <?xml version="1.0" encoding="UTF-8"?>
- <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
- <modelVersion>4.0.0</modelVersion>
- <parent>
- <groupId>org.springframework.boot</groupId>
- <artifactId>spring-boot-starter-parent</artifactId>
- <version>2.1.7.RELEASE</version>
- <relativePath/> <!-- lookup parent from repository -->
- </parent>
-
- <groupId>com.es</groupId>
- <artifactId>es</artifactId>
- <version>0.0.1-SNAPSHOT</version>
- <name>es</name>
- <description>Demo project for Spring Boot</description>
-
- <properties>
- <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
- <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
- <java.version>1.8</java.version>
- </properties>
-
- <dependencies>
- <dependency>
- <groupId>org.springframework.boot</groupId>
- <artifactId>spring-boot-starter</artifactId>
- </dependency>
- <dependency>
- <groupId>org.springframework.boot</groupId>
- <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
- </dependency>
- <dependency>
- <groupId>org.springframework.boot</groupId>
- <artifactId>spring-boot-starter-web</artifactId>
- </dependency>
- <dependency>
- <groupId>org.springframework.boot</groupId>
- <artifactId>spring-boot-starter-test</artifactId>
- <scope>test</scope>
- </dependency>
-
- <dependency>
- <groupId>org.springframework.boot</groupId>
- <artifactId>spring-boot-starter-test</artifactId>
- <scope>test</scope>
- </dependency>
- </dependencies>
-
- <build>
- <plugins>
- <plugin>
- <groupId>org.springframework.boot</groupId>
- <artifactId>spring-boot-maven-plugin</artifactId>
- </plugin>
- </plugins>
- </build>
-
- </project>
(3)创建实体
映射---注解
Spring Data通过注解来声明字段的映射属性,有下面的三个注解:
-
@Document
作用在类,标记实体类为文档对象,一般有两个属性<ul><li> <p>indexName:对应索引库名称</p> </li> <li> <p>type:对应在索引库中的类型</p> </li> <li> <p>shards:分片数量,默认5</p> </li> <li> <p>replicas:副本数量,默认1</p> </li> </ul></li> <li> <p><strong><code>@Id</code></strong> 作用在成员变量,标记一个字段作为id主键</p> </li> <li> <p><strong><code>@Field</code> </strong>作用在成员变量,标记为文档的字段,并指定字段映射属性:</p> <ul><li> <p>type:字段类型,是是枚举:FieldType,可以是text、long、short、date、integer、object等</p> <ul><li> <p>text:存储数据时候,会自动分词,并生成索引</p> </li> <li> <p>keyword:存储数据时候,不会分词建立索引</p> </li> <li> <p>Numerical:数值类型,分两类</p> <ul><li> <p>基本数据类型:long、interger、short、byte、double、float、half_float</p> </li> <li> <p>浮点数的高精度类型:scaled_float</p> <ul><li> <p>需要指定一个精度因子,比如10或100。elasticsearch会把真实值乘以这个因子后存储,取出时再还原。</p> </li> </ul></li> </ul></li> <li> <p>Date:日期类型</p> <ul><li> <p>elasticsearch可以对日期格式化为字符串存储,但是建议我们存储为毫秒值,存储为long,节省空间。</p> </li> </ul></li> </ul></li> <li> <p>index:是否索引,布尔类型,默认是true</p> </li> <li> <p>store:是否存储,布尔类型,默认是false</p> </li> <li> <p>analyzer:分词器名称,这里的<code>ik_max_word</code>即使用ik分词器</p> </li> </ul></li>
- package com.es.bean;
-
- import org.springframework.data.annotation.Id;
- import org.springframework.data.elasticsearch.annotations.Document;
- import org.springframework.data.elasticsearch.annotations.Field;
- import org.springframework.data.elasticsearch.annotations.FieldType;
-
- @Document(indexName = "item", type = "docs", shards = 1, replicas = 0)
- public class Item {
-
- @Id
- private Long id;
- @Field(type = FieldType.Text, analyzer = "ik_max_word")
- private String title; //标题
- @Field(type = FieldType.Keyword)
- private String category;// 分类
- @Field(type = FieldType.Keyword)
- private String brand; // 品牌
- @Field(type = FieldType.Double)
- private Double price; // 价格
- @Field(type = FieldType.Keyword, index = false)
- private String images; // 图片地址
-
-
- public Long getId() {
- return id;
- }
-
- public void setId(Long id) {
- this.id = id;
- }
-
- public String getTitle() {
- return title;
- }
-
- public void setTitle(String title) {
- this.title = title;
- }
-
- public String getCategory() {
- return category;
- }
-
- public void setCategory(String category) {
- this.category = category;
- }
-
- public String getBrand() {
- return brand;
- }
-
- public void setBrand(String brand) {
- this.brand = brand;
- }
-
- public Double getPrice() {
- return price;
- }
-
- public void setPrice(Double price) {
- this.price = price;
- }
-
- public String getImages() {
- return images;
- }
-
- public void setImages(String images) {
- this.images = images;
- }
-
- public Item(Long id, String title, String category, String brand, Double price, String images) {
- this.id = id;
- this.title = title;
- this.category = category;
- this.brand = brand;
- this.price = price;
- this.images = images;
- }
-
- public Item() {
- }
-
- @Override
- public String toString() {
- return "Item{" +
- "id=" + id +
- ", title=‘" + title + ‘‘‘ +
- ", category=‘" + category + ‘‘‘ +
- ", brand=‘" + brand + ‘‘‘ +
- ", price=" + price +
- ", images=‘" + images + ‘‘‘ +
- ‘}‘;
- }
- }
(4)控制层
- package com.es.controller;
-
- import com.es.bean.Item;
- import com.es.dao.ItemRepository;
- import org.elasticsearch.index.query.QueryBuilders;
- import org.elasticsearch.search.aggregations.AggregationBuilders;
- import org.elasticsearch.search.aggregations.bucket.terms.StringTerms;
- import org.elasticsearch.search.aggregations.metrics.avg.InternalAvg;
- import org.elasticsearch.search.sort.SortBuilders;
- import org.elasticsearch.search.sort.SortOrder;
- import org.springframework.beans.factory.annotation.Autowired;
- import org.springframework.data.domain.Page;
- import org.springframework.data.domain.PageRequest;
- import org.springframework.data.domain.Sort;
- import org.springframework.data.elasticsearch.core.ElasticsearchTemplate;
- import org.springframework.data.elasticsearch.core.aggregation.AggregatedPage;
- import org.springframework.data.elasticsearch.core.query.NativeSearchQueryBuilder;
- import org.springframework.web.bind.annotation.GetMapping;
- import org.springframework.web.bind.annotation.RestController;
-
- import java.util.ArrayList;
- import java.util.List;
-
- @RestController
- public class ItemController {
- @Autowired
- ItemRepository itemRepository;
- @Autowired
- ElasticsearchTemplate esTemplate;
-
- /**
- * 创建索引
- * ElasticsearchTemplate中提供了创建索引的API
- */
- @GetMapping("/create/indices")
- public void createIndices() {
- // 创建索引,会根据Item类的@Document注解信息来创建
- esTemplate.createIndex(Item.class);
- // 配置映射,会根据Item类中的id、Field等字段来自动完成映射
- esTemplate.putMapping(Item.class);
- }
-
- /**
- * 删除索引
- */
- @GetMapping("/delete/indices")
- public void deleteIndices() {
- esTemplate.deleteIndex(Item.class);
- // 根据索引名字删除
- //esTemplate.deleteIndex("item");
- }
-
- /**
- * 创建单个索引
- */
- @GetMapping("/add/index")
- public void addIndex() {
- Item item = new Item(1L, "小米手机7", " 手机", "小米", 3499.00, "http://image.baidu.com/13123.jpg");
- itemRepository.save(item);
- }
-
- /**
- * 批量创建索引
- */
- @GetMapping("/add/index/list")
- public void addIndexList() {
- List<Item> list = new ArrayList<Item>();
- list.add(new Item(1L, "小米手机7", "手机", "小米", 3299.00, "http://image.baidu.com/13123.jpg"));
- list.add(new Item(2L, "坚果手机R1", "手机", "锤子", 3699.00, "http://image.baidu.com/13123.jpg"));
- list.add(new Item(3L, "华为META10", "手机", "华为", 4499.00, "http://image.baidu.com/13123.jpg"));
- list.add(new Item(4L, "小米Mix2S", "手机", "小米", 4299.00, "http://image.baidu.com/13123.jpg"));
- list.add(new Item(5L, "荣耀V10", "手机", "华为", 2799.00, "http://image.baidu.com/13123.jpg"));
- // 接收对象集合,实现批量新增
- itemRepository.saveAll(list);
- }
-
- /**
- * 修改索引
- */
- @GetMapping("/update/index")
- public void updateIndex() {
- Item item = new Item(1L, "苹果XSMax", "手机", "小米", 3499.00, "http://image.baidu.com/13123.jpg");
- itemRepository.save(item);
- }
-
- /**
- * 查询所有
- */
- @GetMapping("/find/index")
- public Object queryAll() {
- // 查找所有
- //Iterable<Item> list = this.itemRepository.findAll();
- // 对某字段排序查找所有 Sort.by("price").descending() 降序
- // Sort.by("price").ascending():升序
- Iterable<Item> list = this.itemRepository.findAll(Sort.by("price").ascending());
- for (Item item : list) {
- System.out.println(item);
- }
- return list;
- }
-
-
- /**
- * 价格范围查询
- */
- @GetMapping("/find/index/by/price")
- public Object queryByPriceBetween() {
- List<Item> list = itemRepository.findByPriceBetween(2000.00, 3500.00);
- for (Item item : list) {
- System.out.println("item = " + item);
- }
- return list;
- }
-
- @GetMapping("/find/index/findByCategoryAndPrice")
- public Object findByNameAndPrice() {
- List<Item> list = itemRepository.findByCategoryAndPrice("手机", 3699.00);
- for (Item item : list) {
- System.out.println(item);
- }
-
- return list;
- }
-
- /**
- * match底层是词条匹配
- *
- * @return
- */
- @GetMapping("/find/index/matchQuery")
- public Object matchQuery() {
- //构建查询条件
- NativeSearchQueryBuilder builder = new NativeSearchQueryBuilder();
- //添加分词查询
- builder.withQuery(QueryBuilders.matchQuery("title", "华为"));
- // 查询 自动分页 ,默认查找第一页的10条数据
- Page<Item> list = itemRepository.search(builder.build());
- //总条数
- System.out.println(list.getTotalElements());
- for (Item it : list) {
- System.out.println(it);
- }
- return list;
- }
-
- /**
- * termQuery
- *
- * @return
- */
- @GetMapping("/find/index/termQuery")
- public Object termQuery() {
- // 查询条件生成器
- NativeSearchQueryBuilder queryBuilder = new NativeSearchQueryBuilder();
- queryBuilder.withQuery(QueryBuilders.termQuery("price", 3499.00));
- // 查询 自动分页 ,默认查找第一页的10条数据
- Page<Item> list = itemRepository.search(queryBuilder.build());
- for (Item it : list) {
- System.out.println(it);
- }
- return list;
- }
-
- /**
- * booleanQuery
- *
- * @return
- */
- @GetMapping("/find/index/booleanQuery")
- public Object booleanQuery() {
- NativeSearchQueryBuilder queryBuilder = new NativeSearchQueryBuilder();
- queryBuilder.withQuery(QueryBuilders.boolQuery().must(QueryBuilders.matchQuery("title", "华为")).must(QueryBuilders.matchQuery("brand", "华为")));
- //查找
- Page<Item> list = itemRepository.search(queryBuilder.build());
- System.out.println("总条数:" + list.getTotalElements());
- for (Item it : list) {
- System.out.println(it);
- }
- return list;
- }
-
- /**
- * 模糊查询
- *
- * @return
- */
- @GetMapping("/find/index/fuzzyQuery")
- public Object fuzzyQuery() {
- NativeSearchQueryBuilder queryBuilder = new NativeSearchQueryBuilder();
- queryBuilder.withQuery(QueryBuilders.fuzzyQuery("title", "faceoooo"));
- Page<Item> list = itemRepository.search(queryBuilder.build());
- System.out.println("总条数:" + list.getTotalElements());
- for (Item it : list) {
- System.out.println(it);
- }
- return list;
-
- }
-
- /**
- * 分页查询
- *
- * @return
- */
- @GetMapping("/find/index/pageSearch")
- public Object pageSearch() {
- NativeSearchQueryBuilder queryBuilder = new NativeSearchQueryBuilder();
- queryBuilder.withQuery(QueryBuilders.termQuery("category", "手机"));
- //分页
- int page = 0;
- int size = 3;
- queryBuilder.withPageable(PageRequest.of(page, size));
- //搜索
- Page<Item> page1 = itemRepository.search(queryBuilder.build());
- //总条数
- System.out.println("总条数:" + page1.getTotalElements());
- //总页数
- System.out.println(page1.getTotalPages());
- // 当前页
- System.out.println(page1.getNumber());
- //每页大小
- System.out.println(page1.getSize());
- //所有数据
- for (Item item : page1) {
- System.out.println(item);
- }
- return page1;
- }
-
- /**
- * 排序查询
- */
- @GetMapping("/find/index/searchAndSort")
- public void searchAndSort() {
- // 构建查询条件
- NativeSearchQueryBuilder queryBuilder = new NativeSearchQueryBuilder();
- // 添加基本分词查询
- queryBuilder.withQuery(QueryBuilders.termQuery("category", "手机"));
-
- // 排序
- queryBuilder.withSort(SortBuilders.fieldSort("price").order(SortOrder.ASC));
-
- // 搜索,获取结果
- Page<Item> items = this.itemRepository.search(queryBuilder.build());
- // 总条数
- long total = items.getTotalElements();
- System.out.println("总条数 = " + total);
-
- for (Item item : items) {
- System.out.println(item);
- }
- }
-
- /**
- * 聚合查询
- * 聚合为桶bucket--分组--类似group by
- * 桶就是分组,比如这里我们按照品牌brand进行分组:
- */
- @GetMapping("/find/index/searchAgg")
- public Object searchAgg(){
- NativeSearchQueryBuilder queryBuilder = new NativeSearchQueryBuilder();
- // 不查询任何结果
- //queryBuilder.withSourceFilter(new FetchSourceFilter(new String[]{""}, null));
- // 1、添加一个新的聚合,聚合类型为terms,聚合名称为brands,聚合字段为brand
- queryBuilder.addAggregation(
- AggregationBuilders.terms("brands").field("brand"));
- // 2、查询,需要把结果强转为AggregatedPage类型
- AggregatedPage<Item> aggPage = (AggregatedPage<Item>) this.itemRepository.search(queryBuilder.build());
- // 3、解析
- // 3.1、从结果中取出名为brands的那个聚合,
- // 因为是利用String类型字段来进行的term聚合,所以结果要强转为StringTerm类型
- StringTerms agg = (StringTerms) aggPage.getAggregation("brands");
- // 3.2、获取桶
- List<StringTerms.Bucket> buckets = agg.getBuckets();
- // 3.3、遍历
- for (StringTerms.Bucket bucket : buckets) {
- // 3.4、获取桶中的key,即品牌名称
- System.out.println(bucket.getKeyAsString());
- // 3.5、获取桶中的文档数量
- System.out.println(bucket.getDocCount());
- }
- return buckets;
- }
-
- /**
- * 嵌套聚合,求平均值---度量
- * 需求:求桶--分组,每个品牌手机的平均价格
- * 思路:(分组求桶) + 求平均值(度量)
- */
- @GetMapping("/find/index/subAgg")
- public Object subAgg() {
- NativeSearchQueryBuilder queryBuilder1 = new NativeSearchQueryBuilder();
- queryBuilder1.addAggregation(AggregationBuilders.terms("brands").field("brand")
- .subAggregation(AggregationBuilders.avg("priceAvg").field("price")));
-
- AggregatedPage<Item> aggregatedPage = (AggregatedPage<Item>) itemRepository.search(queryBuilder1.build());
-
- StringTerms brands = (StringTerms) aggregatedPage.getAggregation("brands");
-
- List<StringTerms.Bucket> buckets = brands.getBuckets();
- for (StringTerms.Bucket bu : buckets) {
- System.out.print(bu.getKeyAsString() + " " + bu.getDocCount() + " ");
-
- InternalAvg avg = (InternalAvg) bu.getAggregations().asMap().get("priceAvg");
- System.out.println(avg.getValue());
- }
- return buckets;
- }
- }
(5)Repository接口
- package com.es.dao;
-
- import com.es.bean.Item;
- import org.springframework.data.elasticsearch.repository.ElasticsearchRepository;
-
- import java.util.List;
-
- /**
- * 接口关系:
- * ElasticsearchRepository --> ElasticsearchCrudRepository --> PagingAndSortingRepository --> CrudRepository
- */
- public interface ItemRepository extends ElasticsearchRepository<Item,Long> {
-
- /**
- * 根据价格区间查询
- * @param price1
- * @param price2
- * @return
- */
- List<Item> findByPriceBetween(double price1, double price2);
-
- List<Item> findByCategoryAndPrice(String name, double price);
- }
(6)application.properties配置
- spring.data.elasticsearch.repositories.enabled=true
- spring.data.elasticsearch.cluster-name=zzq-es
- spring.data.elasticsearch.cluster-nodes=192.168.1.16:9300
(7)启动类
- package com.es;
-
- import org.springframework.boot.SpringApplication;
- import org.springframework.boot.autoconfigure.SpringBootApplication;
-
- @SpringBootApplication
- public class EsApplication {
-
- public static void main(String[] args) {
- SpringApplication.run(EsApplication.class, args);
- }
-
- }
四、总结
(1)端口问题
redis: 6379
mq: 浏览器访问 6181
代码访问 61616
es: 浏览器访问 9200
代码访问 9300
(2)自定义方法
Keyword | Sample |
And | findByNameAndPrice findBy属性名1And属性名2 |
Or | findByNameOrPrice |
Is | findByName |
Not | findByNameNot |
Between | findByPriceBetween |
LessThanEqual | findByPriceLessThan |
GreaterThanEqual | findByPriceGreaterThan |
Before | findByPriceBefore |
After | findByPriceAfter |
Like | findByNameLike |
StartingWith | findByNameStartingWith |
Contains/Containing | findByNameContaining |
In | findByNameIn(Collection<String>names) |
NotIn | findByNameNotIn(Collection<String>names) |
Near | findByStoreNear |
True | findByAvailableTrue |
False | findByAvailableFalse |
OrderBy | findByAvailableTrueOrderByNameDesc |
例如
- public interface ItemRepository extends ElasticsearchRepository<Item,Long> {
-
- /**
- * 根据价格区间查询
- * @param price1
- * @param price2
- * @return
- */
- List<Item> findByPriceBetween(double price1, double price2);
- }
(3)基本概念
Elasticsearch也是基于Lucene的全文检索库,本质也是存储数据,很多概念与关系型数据相似。
对比关系:
索引库(indices)--------------------------------Databases 数据库
- 类型(type)-----------------------------Table 数据表
- 文档(Document)----------------Row 行
- 字段(Field)-------------------Columns 列
概念 | 说明 |
索引库(indices) | indices是index的复数,代表许多的索引, |
类型(type) | 类型是模拟mysql中的table概念,一个索引库下可以有不同类型的索引,比如商品索引,订单索引,其数据格式不同。不过这会导致索引库混乱,因此未来版本中会移除这个概念 |
文档(document) | 存入索引库原始的数据。比如每一条商品信息,就是一个文档 |
字段(field) | 文档中的属性 |
映射配置(mappings) | 字段的数据类型、属性、是否索引、是否存储等特性 |
另外,在Elasticsearch有一些集群相关的概念:
索引集(Indices,index的复数):逻辑上的完整索引
分片(shard):数据拆分后的各个部分
副本(replica):每个分片的复制
要注意的是:Elasticsearch本身就是分布式的,因此即便你只有一个节点,Elasticsearch默认也会对你的数据进行分片和副本操作,当你向集群添加新数据时,数据也会在新加入的节点中进行平衡。
github : https://github.com/2014team/elasticsearch
推荐文章:https://blog.csdn.net/weixin_42633131/article/details/82902812
以上是关于SprignBoot整合Spring Data Elasticsearch的主要内容,如果未能解决你的问题,请参考以下文章
《SpringCloud 从入门到入土 》 第二章:微服务构建:Spring Boot
《SpringCloud 从入门到入土 》 第二章:微服务构建:Spring Boot
Spring整合HibernateHibernate JPASpring Data JPASpring Data Redis
Spring整合HibernateHibernate JPASpring Data JPASpring Data Redis