elasticsearch 7.6.2 - 索引管理
Posted k↑
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了elasticsearch 7.6.2 - 索引管理相关的知识，希望对你有一定的参考价值。
小伙伴们，你们好呀，我是老寇，跟我一起学习es 7.6.2
目录
一.创建索引及参数解析
1.创建索引 (语法及例子)
1).新建索引语法
PUT 索引名称
"setting" : ... ,
"mapping" :
"properties" :
"field1" :
"type" : "keyword"
,
"field2" :
"type" : "keyword"
...
...
...
2).新建消息索引例子
PUT msg_202203
"settings":
"index":
"refresh_interval": "60s",
"number_of_shards": "5",
"analysis":
"filter":
"laokou-remote-synonym":
"dynamic_reload": "true",
"interval": "60",
"type": "dynamic_synonym",
"synonyms_path": "http://localhost:9048/laokou-service/synonym"
,
"laokou-pinyin":
"lowercase": "true",
"keep_original": "true",
"remove_duplicated_term": "true",
"keep_first_letter": "true",
"keep_separate_first_letter": "false",
"type": "pinyin",
"limit_first_letter_length": "16",
"keep_full_pinyin": "true"
,
"analyzer":
"ik-search-pinyin":
"filter": ["laokou-pinyin", "word_delimiter"],
"char_filter": ["html_strip"],
"type": "custom",
"tokenizer": "ik_smart"
,
"ik-search-synonym":
"filter": ["laokou-remote-synonym", "lowercase", "asciifolding"],
"char_filter": ["html_strip"],
"type": "custom",
"tokenizer": "ik_smart"
,
"ik-index-synonym":
"filter": ["laokou-remote-synonym", "lowercase", "asciifolding"],
"char_filter": ["html_strip"],
"type": "custom",
"tokenizer": "ik_max_word"
,
"number_of_replicas": "1"
,
"aliases":
"msg":
,
"mappings":
"dynamic": "true",
"properties":
"sendId":
"type": "long"
,
"data":
"eager_global_ordinals": true,
"search_analyzer": "ik-search-synonym",
"fielddata": true,
"analyzer": "ik-index-synonym",
"boost": 100,
"type": "text",
"fields":
"data-pinyin":
"analyzer": "ik-search-pinyin",
"term_vector": "with_positions_offsets",
"boost": 100,
"type": "text"
,
"type":
"type": "integer"
,
"remark":
"type": "keyword"
,
"fromId":
"type": "long"
,
"createDate":
"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis",
"type": "date"
,
"username":
"type": "keyword"
2.索引参数解析 (着重解析例子中参数含义)
1).setting
setting是索引的配置属性
"settings": #settings配置
"index":
"refresh_interval": "60s", #表示索引刷新频率,频繁刷新索引会降低性能,一般设置为30s~60s;-1表示禁用刷新
"number_of_shards": "5", #index分片数,需要注意的是es7.0默认索引分片调整为1
"analysis": #定制分词器,包含filter、analyzer
"filter":
"laokou-remote-synonym": #同义词分词过滤器名称
"dynamic_reload": "true", #开启动态加载同义词库
"interval": "60", #同步频率,单位为秒
"type": "dynamic_synonym", #同义词分词
"synonyms_path": "http://localhost:9048/laokou-service/synonym" #远程同义词库地址
,
"laokou-pinyin": #拼音分词过滤器名称
"lowercase": "true", #开启小写
"keep_original": "true", #是否保留原始输入 默认值false
"remove_duplicated_term": "true", #是否删除重复项保留索引，默认值false
"keep_first_letter": "true", #是否开启首字母匹配 例如：寇申海 > ksh 默认值true
"keep_separate_first_letter": "false", #保留第一个字母分开，例如：寇申海> k，s，h，默认：false
"type": "pinyin", #拼音分词
"limit_first_letter_length": "16", #设置first_letter结果的最大长度，默认值：16
"keep_full_pinyin": "true" #是否开启全拼 例如寇申海 > kou shen hai 默认值true
,
"analyzer": #分词器配置
"ik-search-pinyin": #自定义查询拼音分词器
"filter": ["laokou-pinyin", "word_delimiter"], #拼音过滤器，word_delimiter 词元分析器（将单词分为字词，例如koushenhai 分成kou shen hai）
"char_filter": ["html_strip"], #字符过滤器，html_filter字符过滤器可删除所有html标签，例如<p>
"type": "custom", #自定义
"tokenizer": "ik_smart" #ik分词中的简单分词器，支持自定义字典，远程字典
,
"ik-search-synonym": #自定义查询同义词分词器
"filter": ["laokou-remote-synonym", "lowercase", "asciifolding"], #同义词过滤器，单词小写过滤器，asciifolding词元过滤器（将不在前127个ASCII字符（“基本拉丁文”Unicode块）中的字母，数字和符号Unicode字符转换为ASCII等效项）
"char_filter": ["html_strip"], #字符过滤器，html_filter字符过滤器删除所有html标签
"type": "custom", #自定义
"tokenizer": "ik_smart" #ik分词的简单分词，支持自定义字典、远程字典
,
"ik-index-synonym": #自定义索引同义词分词器
"filter": ["laokou-remote-synonym", "lowercase", "asciifolding"], #同义词过滤器，单词小写过滤器，asciifolding词元过滤器
"char_filter": ["html_strip"], #字符过滤器，html_filter字符过滤器删除所有html标签
"type": "custom", #自定义
"tokenizer": "ik_max_word" #ik分词中的最大分词器，支持自定义字典，远程字典，例如我是中国人，分成我是，中国，中国人，我是中国人
,
"number_of_replicas": "1" #索引分片的备份数
2).mapping
mapping是指定索引存储文档的字段类型
"mappings":
"dynamic": "true", #动态加载
"properties": #属性
"sendId":
"type": "long"
,
"data":
"eager_global_ordinals": true, #开启后，每次refresh以后即可更新字典，字典常驻内存，减少查询时构建字典的耗时
"search_analyzer": "ik-search-synonym", #查询时 ik-smart分词器
"fielddata": true,
"analyzer": "ik-index-synonym", #建立索引-ik-max分词器
"boost": 100,
"type": "text",
"fields":
"data-pinyin": #因进行拼音查询，需要设置属性字段
"analyzer": "ik-search-pinyin", #自定义拼音查询分词器
"term_vector": "with_positions_offsets", #文档的统计信息
"boost": 100, #权重
"type": "text"
,
"type":
"type": "integer"
,
"remaker":
"type": "keyword"
"fromId":
"type": "long"
,
"createDate":
"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis", #存入date类型，需要进行格式化
"type": "date"
,
"username":
"type": "keyword"
3).aliases
aliases是给索引取别名
aliases:
"msg": #别名名称
4).执行命令(使用kibana执行，没有就安装kibana)
补充： 获取远程同义词库核心代码（详细代码已上传到码云）
@RestController
@RequestMapping("/synonym")
public class SynonymController
private static final Logger log = LoggerFactory.getLogger(SynonymController.class);
@Autowired
private SynonymDao synonymDao;
@Autowired
private RedisUtil redisUtil;
/**
* SimpleDateFormat线程不安全
*/
private static final ThreadLocal<DateFormat> df = ThreadLocal.withInitial(() -> new SimpleDateFormat(DateUtil.DATE_TIME));
/**
* 查询同义词库
* @param request
* @param response
* @return
* @apiNote 小伙伴们，你好呀，我是老寇
* 思考1：一分钟调一次接口，但是如果有很多索引同时调用一个接口，接口的性能会受到影响
* 因此可以引入redis，来分摊mysql查询的压力
*/
@GetMapping
@CrossOrigin
public String text(HttpServletRequest request, HttpServletResponse response)
String result = "";
String eTag = request.getHeader("If-None-Match");
String modified = request.getHeader("If-Modified-Since");
String currentDate = df.get().format(new Date());
String synonymKey = RedisKeyUtil.getSynonymKey();
String dataJson = redisUtil.get(synonymKey);
List<SynonymEntity> list;
if (StringUtils.isNotBlank(dataJson))
list = JSON.parseArray(dataJson,SynonymEntity.class);
else
QueryWrapper<SynonymEntity> queryWrapper = new QueryWrapper<SynonymEntity>().select("value");
list = synonymDao.selectList(queryWrapper);
redisUtil.set(synonymKey,JSON.toJSONString(list),RedisUtil.HOUR_ONE_EXPIRE);
if (CollectionUtils.isEmpty(list))
return null;
List<String> valueList = list.stream().map(SynonymEntity::getValue).collect(Collectors.toList());
log.info("加载ik同义词，上次同义词:,上次修改时间：,当前日期:",eTag,modified,currentDate);
if(!valueList.isEmpty())
// 获取数据库同义词
StringBuilder words = new StringBuilder();
for (String synonym : valueList)
// 可以使用“=>”方式或者直接“,”分割形式，自行考虑应用场景
words.append(synonym);
words.append("\\n");
modified = currentDate;
result = words.toString();
//更新时间
response.setHeader("Last-Modified", modified);
response.setHeader("ETag", String.valueOf(list.size()));
response.setHeader("Content-Type", "text/plain");
return result;
二.批量导入数据
1.批量导入(语法及例子)
1).批量导入语法
POST /_bulk
"action": "metadata"
"data"
POST /_bulk
"create":"_index":"索引名称" #填写索引名称
"field1":"value1","field2":"value2" #实体类的数据2).批量导入例子（因为后面的教程会用到一些数据，索性就多导入几条数据）
POST /_bulk
"create":"_index":"msg_202203"
"createDate":1614129064000,"data":"谷神不死，是谓玄牝。玄牝之门，是谓天地根。绵绵若存，用之不勤。","fromId":"1341620898007281665","remark":"群聊","sendId":"1363109342432645122","type":2,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1614060893000,"data":"天地不仁，以万物為芻狗；圣人不仁，以百姓為芻狗。","fromId":"1341620898007281665","remark":"群聊","sendId":"1363109342432645122","type":2,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1614059233000,"data":"道沖，而用之或不盈。淵兮，似万物之宗；湛兮，似或存。吾不知誰之子，象帝之先。","fromId":"1341620898007281665","remark":"群聊","sendId":"1363109342432645122","type":2,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1613826379000,"data":"是以圣人之治，虛其心，實其腹，弱其志，強其骨。常使民無知無欲。使夫智者不敢為也。為無為，則無不治。","fromId":"1341620898007281665","remark":"群聊","sendId":"1363109342432645122","type":2,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1613826260000,"data":"不尚賢，使民不爭；不貴難得之貨，使民不為盜；不見可欲，使民心不亂。","fromId":"1341620898007281665","remark":"群聊","sendId":"1363109342432645122","type":2,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1613825567000,"data":"夫唯不居，是以不去。","fromId":"1341620898007281665","remark":"群聊","sendId":"1363109342432645122","type":2,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1613825555000,"data":"是以圣人处无为之事，行不言之教，万物作焉而不辞，生而不有，为而不恃，功成而不居。","fromId":"1341620898007281665","remark":"群聊","sendId":"1363109342432645122","type":2,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1613825534000,"data":"天下皆知美之为美，斯恶矣；皆知善之为善，斯不善矣。","fromId":"1341620898007281665","remark":"群聊","sendId":"1363109342432645122","type":2,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1613825140000,"data":"故常无欲，以观其妙；常有欲，以观其徼。","fromId":"1341620898007281665","remark":"群聊","sendId":"1363109342432645122","type":2,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1613551503000,"data":"此两者，同出而异名，同谓之玄，玄之又玄，众妙之门。","fromId":"1341620898007281665","remark":"群聊","sendId":"1363109342432645122","type":2,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1613377999000,"data":"无名，天地之始，有名，万物之母。","fromId":"1341620898007281665","remark":"群聊","sendId":"1363109342432645122","type":2,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1613375454000,"data":"道可道，非常道；名可名，非常名。","fromId":"1341620898007281665","remark":"群聊","sendId":"1363109342432645122","type":2,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1612854931000,"data":"真常应物，真常得性；常应常静，常清静矣。如此清静，渐入真道；既入真道，名为得道；虽名得道，实无所得；为化众生，名为得道；能悟之者，可传圣道。","fromId":"1341620898007281665","remark":"群聊","sendId":"1363109342432645122","type":2,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1612854896000,"data":"夫，人神好清，而心扰之；人心好静，而欲牵之。常能遣其欲，而心自静；澄其心，而神自清；自然六欲不生，三毒消灭。所以不能者，为心未澄，欲未遣也，能遣之者：内观其心，心无其心；外观其形，形无其形；远观其物，物无其物；三者既无，唯见於空。观空亦空，空无所空；所空既无，无无亦无；无无既无，湛然常寂。寂无所寂，欲岂能生；欲既不生，即是真静。","fromId":"1341620898007281665","remark":"私聊","sendId":"1341623527018004481","type":1,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1612854529000,"data":"夫道者：有清有浊，有动有静；天清地浊，天动地静；男清女浊，男动女静；降本流末，而生万物。清者浊之源，动者静之基；人能常清静，天地悉皆归。","fromId":"1341620898007281665","remark":"群聊","sendId":"1363109342432645122","type":2,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1612854482000,"data":"大道无形，生育天地；大道无情，运行日月；大道无名，长养万物；吾不知其名，强名曰道。","fromId":"1341620898007281665","remark":"私聊","sendId":"1341623527018004481","type":1,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1635830300000,"data":"好东西不用你去记，它自会留下很深的印象。","fromId":"1341623527018004481","remark":"群聊","sendId":"1363109342432645122","type":2,"username":"test"
"create":"_index":"msg_202203"
"createDate":1635814452000,"data":"最困难的事情就是认识自己。","fromId":"1341620898007281665","remark":"群聊","sendId":"1363109342432645122","type":2,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1627093178000,"data":"上善若水。","fromId":"1341620898007281665","remark":"私聊","sendId":"1341623527018004481","type":1,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1627090427000,"data":"戒为良药，以戒为师。","fromId":"1341620898007281665","remark":"私聊","sendId":"1341623527018004481","type":1,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1629213470000,"data":"有时候，不是因为自己不坚强，而是，总想着有一个人关心一下自己，鼓励自己。","fromId":"1341623527018004481","remark":"私聊","sendId":"1341620898007281665","type":1,"username":"test"
"create":"_index":"msg_202203"
"createDate":1628999314000,"data":"祝你的生日天天快乐,永远都幸福.在新的一年里感情好,身体好,事业好,对你的朋友们要更好!","fromId":"1341623527018004481","remark":"群聊","sendId":"1363109342432645122","type":2,"username":"test"
"create":"_index":"msg_202203"
"createDate":1628999280000,"data":"绿色是生命的颜色，绿色的浪漫是生命的浪漫。因此，我选择了这个绿色的世界，馈赠给你的生日。愿你充满活力，青春常在。","fromId":"1341623527018004481","remark":"私聊","sendId":"1341620898007281665","type":1,"username":"test"
"create":"_index":"msg_202203"
"createDate":1628999215000,"data":"羡慕你的生日是这样浪漫，充满诗情画意，只希望你的每一天都快乐、健康、美丽，生命需要奋斗、创造、把握!生日快乐","fromId":"1341620898007281665","remark":"群聊","sendId":"1363109342432645122","type":2,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1628999188000,"data":"青春的树越长赵葱茏，生命的花就越长越艳丽。在你生日的这一天，请接受我对你深深的祝福。愿这独白，留在你生命的扉页;愿这切切祈盼，带给你新的幸福。","fromId":"1341620898007281665","remark":"私聊","sendId":"1341623527018004481","type":1,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1628998224000,"data":"今夜，惊悉你的生日。窗外的风带上我的祝福，祝愿你在新的一年里心想事成花容月貌而且又乖又可爱","fromId":"1341623527018004481","remark":"私聊","sendId":"1341620898007281665","type":1,"username":"test"
"create":"_index":"msg_202203"
"createDate":1628998136000,"data":"长长的距离，长长的线，连着长长的思念。远远的空间，久久的时间，剪不断远方的掂念!祝你生日快乐","fromId":"1341620898007281665","remark":"群聊","sendId":"1363109342432645122","type":2,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1628998098000,"data":"虽然不能陪你度过这特别的日子，但我的祝福依旧准时送上：在你缤纷的人生之旅中，心想事成!生日快乐!","fromId":"1341620898007281665","remark":"私聊","sendId":"1341623527018004481","type":1,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1628997414000,"data":"愿我的祝福，如一缕灿烂的阳光，在您的眼里流尚，生日快乐!","fromId":"1341623527018004481","remark":"私聊","sendId":"1341620898007281665","type":1,"username":"test"
"create":"_index":"msg_202203"
"createDate":1628997401000,"data":"又是你的生日了，虽然残破的爱情让我们彼此变得陌生，然而我从未忘你的生日，好多年了，每一个生日我们都没忘记给对方祝福，希望这一生都拥有这友情，真心祝你生日快乐!","fromId":"1341620898007281665","remark":"群聊","sendId":"1363109342432645122","type":2,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1628997381000,"data":"祝我漂亮的、乐观的、热情的、健康自信的、布满活力的大朋友——妈妈，生日快乐!","fromId":"1341620898007281665","remark":"私聊","sendId":"1341623527018004481","type":1,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1628997362000,"data":"你用厚重的年轮，编成一册散发油墨清香的日历;年年，我都会在日历的这一天上，带着崇高的敬意，祝福你的生日。","fromId":"1341623527018004481","remark":"私聊","sendId":"1341620898007281665","type":1,"username":"test"
"create":"_index":"msg_202203"
"createDate":1628950044000,"data":"远方的朋友，我衷心的祝你：年年好“薪”情，岁岁好“钱”景，天天好福气，时时好运来，做个“四好”新人。","fromId":"1341623527018004481","remark":"私聊","sendId":"1341620898007281665","type":1,"username":"test"
"create":"_index":"msg_202203"
"createDate":1628949929000,"data":"年年岁岁花相似，岁岁年年人不同。醒来惊觉不是梦，眉间皱纹又一重。","fromId":"1341620898007281665","remark":"私聊","sendId":"1341623527018004481","type":1,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1628573383000,"data":"烛光支支辉煌，寿诞岁岁吉利。童颜鹤发逢盛世，百年不老福乐绵。麻姑献寿奉蟠桃，我来贺寿送祝愿：祝你与天地比寿，愿你与日月同光！","fromId":"1341620898007281665","remark":"私聊","sendId":"1341623527018004481","type":1,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1628573219000,"data":"黄金白银，每一两都坚强无比，珍珠美玉，每一粒都完美无瑕，钻石流星，每一颗都永恒流传，可是这些都比不过今天你的心，因为寿星老最高贵，生日快乐。","fromId":"1341623527018004481","remark":"私聊","sendId":"1341620898007281665","type":1,"username":"test"
"create":"_index":"msg_202203"
"createDate":1628557084000,"data":"懒人无法享受休息之乐。","fromId":"1341623527018004481","remark":"私聊","sendId":"1341620898007281665","type":1,"username":"test"
"create":"_index":"msg_202203"
"createDate":1628557013000,"data":"伟大的事业，需要决心，能力，组织和责任感。","fromId":"1341620898007281665","remark":"私聊","sendId":"1341623527018004481","type":1,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1628556795000,"data":"学而不思则罔，思而不学则殆。","fromId":"1341623527018004481","remark":"私聊","sendId":"1341620898007281665","type":1,"username":"test"
"create":"_index":"msg_202203"
"createDate":1628556767000,"data":"只有把抱怨环境的心情，化为上进的力量，才是成功的保证。","fromId":"1341620898007281665","remark":"私聊","sendId":"1341623527018004481","type":1,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1628556726000,"data":"最具挑战性的挑战莫过于提升自我。","fromId":"1341623527018004481","remark":"私聊","sendId":"1341620898007281665","type":1,"username":"test"
"create":"_index":"msg_202203"
"createDate":1628556628000,"data":"老寇，生日快乐","fromId":"1341620898007281665","remark":"私聊","sendId":"1341623527018004481","type":1,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1624878509000,"data":"民之饥，以其上食税之多，是以饥。","fromId":"1341620898007281665","remark":"群聊","sendId":"1363109342432645122","type":2,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1624878463000,"data":"强大处下，柔弱处上。","fromId":"1341620898007281665","remark":"私聊","sendId":"1341623527018004481","type":1,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1624874411000,"data":"天之道，其犹张弓与？高者抑下，下者举之，有余者损之，不足者补之。","fromId":"1341620898007281665","remark":"私聊","sendId":"1341623527018004481","type":1,"username":"admin"
"create":"_index":"msg_202203"
"createDate":1623390047000,"data":"顺其自然，无为而治。","fromId":"1341620898007281665","remark":"群聊","sendId":"1363109342432645122","type":2,"username":"admin"3).等待数据批量导入（连接kibana进行批量导入）
注：因为是异步，可能在kibana执行完了，数据并没有提交到es中，需要等待一会儿
三、查询索引
1.查询mapping
GET /索引/_mapping
GET /msg_202203/_mapping2.查询settings
GET /索引/_settings
GET /msg_202203/_settings四、修改索引
PUT /msg_202203/_settings
"index":
"refresh_interval": "-1",
"number_of_shards": "5",
"number_of_replicas": "1"
五、删除索引
DELETE /msg_202203
DELETE /msg_*以上是关于elasticsearch 7.6.2 - 索引管理的主要内容，如果未能解决你的问题，请参考以下文章
使用 FVH 突出显示 ElasticSearch 7.6.2 的正确设置是啥？
springboot 2.0集成elasticsearch 7.6.2(集群)
centos7 - elasticsearch 7.6.2集群
elasticsearch创建索引报[type=mapper_parsing_exception, reason=Failed to parse mapping [_doc]......
centos 7.5 安装ELK elasticsearch-7.6.2单点服务器+logstash+filebeat+kibana7.6.2安装设置