java 整合elk 日志

Posted 2023-03-11 Surperw985

1、搭建elk

使用docker-compose 搭建

一、编写docker-compose.yml

version: "3"
services:
  elasticsearch:
    image: elasticsearch:7.5.1
    container_name: elasticsearch
    environment:
      - "cluster.name=elasticsearch" #设置集群名称为elasticsearch
      - "discovery.type=single-node" #以单一节点模式启动
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m" #设置使用jvm内存大小
    volumes:
      - /opt/xx/docker/elk/elasticsearch/plugins:/opt/jiyifan/docker/elk/elasticsearch/plugins #插件文件挂载
      - /opt/xx/docker/elk/elasticsearch/data:/opt/jiyifan/docker/elk/elasticsearch/data #数据文件挂载
    ports:
      - 9200:9200
  kibana:
    image: kibana:7.5.1
    container_name: kibana
    depends_on:
      - elasticsearch #kibana在elasticsearch启动之后再启动
    environment:
      - ELASTICSEARCH_URL=http://elasticsearch:9200 #设置访问elasticsearch的地址
    ports:
      - 5601:5601
  logstash:
    image: logstash:7.5.1
    container_name: logstash
    volumes:
      - /opt/xx/docker/elk/logstash/logstash-springboot.conf:/usr/share/logstash/pipeline/logstash.conf #挂载logstash的配置文件
    depends_on:
      - elasticsearch #kibana在elasticsearch启动之后再启动
    links:
      - elasticsearch:es #可以用es这个域名访问elasticsearch服务
    ports:
      - 4560:4560

二、logstash-springboot.conf 文件

*.conf 文件名称跟上方docker-compose.yml文件中的 logstash volumes 路劲一致

执行命令:

mkdir -p /opt/xx/docker/elk/logstash/

cd /opt/xx/docker/elk/logstash/

vim logstash-springboot.conf

将下方cv进去

input 
  tcp 
    mode => "server"
    host => "0.0.0.0"
    port => 4560
    codec => json_lines
  

output 
  elasticsearch 
    hosts => "es:9200"
    index => "springboot-logstash-%+YYYY.MM.dd"
  

三、启动命令

mkdir -p /opt/jiyifan/docker/elk/elasticsearch

cd /opt/jiyifan/docker/elk

vim docker-compose.yml

将上面yml文件cv进去

docker-compose up -d

四、检查是否成功

docker ps 查看三个镜像是否都已经创建

127.0.0.1:9200 es 地址

127.0.0.1:5601 kibana 地址

二、java 整合elk(springboot 项目)

resources 目录下面创建 logback.xml 文件

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/base.xml" />

<appender name="LOGSTASH" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
    <!-- 这里需要修改logstash地址 -->
    <destination>127.0.0.1:4560</destination>
    <!-- 日志输出编码 -->
    <encoder charset="UTF-8"
             class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
        <providers>
            <timestamp>
                <timeZone>UTC</timeZone>
            </timestamp>
            <pattern>
                <pattern>
                    
                    "logLevel": "%level",
                    "serviceName": "服务名称要替换方便搜索",
                    "pid": "$PID:-",
                    "thread": "%thread",
                    "class": "%logger40",
                    "rest": "%message"
                    
                </pattern>
            </pattern>
        </providers>
    </encoder>
</appender>
<root level="INFO">
        <appender-ref ref="CONSOLE"/>
        <appender-ref ref="FILE"/>
        <appender-ref ref="LOGSTASH"/>
</root>
</configuration>

增加依赖

    <dependency>
                <groupId>net.logstash.logback</groupId>
                <artifactId>logstash-logback-encoder</artifactId>
                <version>5.2</version>
     </dependency>

测试代码

@RestController
@RequestMapping("/elk")
@Slf4j
public class TestController 
    @GetMapping
    public AjaxResult connectionElk()

        log.info("连接es 成功");
        throw  new RuntimeException("测试es报错");
//    return AjaxResult.success();
    

三、kabana 创建logstash索引

这里有个坑必须这个索引里面有值才能在这里展示出来 一定要执行一下上面的方法

四、成果展示

此处可以搜索