SpringBoot项目日志输出为JSON格式,方便Elastic采集

Posted 北亮bl

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了SpringBoot项目日志输出为JSON格式,方便Elastic采集相关的知识,希望对你有一定的参考价值。

这段时间,使用K8S部署项目,同时使用Elastic采集日志并提供给开发检索,使用过程中,经常出现换行日志无法完整采集和展示的问题。
比如抛出的Exception会有换行,被ES采集后,变成多条日志,极其不方便检索和排错。

经运维提示,并参考 https://groups.google.com/g/fluent-bit/c/XoOibAxGGeI
改用json输出项目日志,本地调试依然使用旧日志格式。
注:本文基于:
spring-boot-starter:2.4.11 或 spring-boot-starter:2.3.10.RELEASE

1、打开项目pom.xml,添加依赖:

<dependency>
    <groupId>net.logstash.logback</groupId>
    <artifactId>logstash-logback-encoder</artifactId>
    <version>6.4</version>
</dependency>

2、在项目的resources目录下,添加文件 logback-spring.xml
注:文件名不允许是 logback.xml ,否则会报错:
no applicable action for [springProfile], current ElementPath is [[configuration][springProfile]]
文件内容参考:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <include resource="org/springframework/boot/logging/logback/defaults.xml"/>
    <include resource="org/springframework/boot/logging/logback/console-appender.xml"/>
    <appender name="JsonAppender" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="net.logstash.logback.encoder.LogstashEncoder">
            <fieldNames>
                <timestamp>dt</timestamp>
                <version>[ignore]</version>
                <levelValue>[ignore]</levelValue>
                <stackTrace>exp</stackTrace>
            </fieldNames>
            <!-- This 'fixes' double quote problem for elastic -->
            <jsonFactoryDecorator class="net.logstash.logback.decorate.CharacterEscapesJsonFactoryDecorator">
                <escape>
                    <targetCharacterCode>34</targetCharacterCode>
                    <escapeSequence>'</escapeSequence>
                </escape>
            </jsonFactoryDecorator>
            <!-- This shortens extremely long stacktraces. Not always needed -->
            <throwableConverter class="net.logstash.logback.stacktrace.ShortenedThrowableConverter">
                <maxDepthPerThrowable>30</maxDepthPerThrowable>
                <maxLength>2048</maxLength>
                <shortenedClassNameLength>20</shortenedClassNameLength>
                <exclude>sun\\.reflect\\..*\\.invoke.*</exclude>
                <exclude>net\\.sf\\.cglib\\.proxy\\.MethodProxy\\.invoke</exclude>
                <rootCauseFirst>true</rootCauseFirst>
                <inlineHash>true</inlineHash>
            </throwableConverter>
        </encoder>
    </appender>
    <root level="INFO">
        <!-- 配置文件的:spring.profiles.active为local时,本地输出标准格式 -->
        <springProfile name="local">
            <appender-ref ref="CONSOLE"/>
        </springProfile>
        <!-- 其它环境输出json格式 -->
        <springProfile name="!local">
            <appender-ref ref="JsonAppender"/>
        </springProfile>
    </root>
</configuration>

OK,构建项目就可以看到输出的都是JSON格式了:
正常的INFO日志如下:

"dt":"2022-01-19T14:46:34.762+08:00","message":"请求:/v2/test | 返回='status':'0','success':true","logger_name":"beinet.cn.web.advice.WebResponseBodyAdvice","thread_name":"http-nio-80-exec-9","level":"INFO"

抛出的异常日志如下:

"dt":"2022-01-19T14:53:30.199+08:00","message":"testErr:","logger_name":"beinet.cn.weblog.IndexController","thread_name":"http-nio-8881-exec-2","level":"ERROR","exp":"<#56381f86> j.l.NumberFormatException: null\\r\\n\\tat java.lang.Integer.parseInt(Integer.java:614)\\r\\n\\tat java.lang.Integer.parseInt(Integer.java:770)...\\r\\n"

另:项目里使用了MyBatis-plus,需要输出SQL明细日志,之前是在配置里添加:
mybatis-plus:
  configuration:
    log-impl: org.apache.ibatis.logging.stdout.StdOutImpl # 输出SQL日志

这种是直接输出Console日志,也会影响采集,把它也改成JSON格式:

mybatis-plus:
  configuration:
    #log-impl: org.apache.ibatis.logging.stdout.StdOutImpl # 输出SQL日志
# mybatis-plus 输出sql日志在logback里
logging.level:
  beinet.cn.dal: debug  # 这是mybatis-plus的mapper类所在的package名

输出参考:

"dt":"2022-01-19T14:51:55.790+08:00","message":"==>  Preparing: select * from appinfo where app_key=?","logger_name":"beinet.cn.dal.AppInfoMapper.findByAppKey","thread_name":"http-nio-8881-exec-1","level":"DEBUG","tags":["MYBATIS"]
"dt":"2022-01-19T14:51:55.829+08:00","message":"==> Parameters: abc(String)","logger_name":"beinet.cn.dal.AppInfoMapper.findByAppKey","thread_name":"http-nio-8881-exec-1","level":"DEBUG","tags":["MYBATIS"]
"dt":"2022-01-19T14:51:55.929+08:00","message":"<==      Total: 1","logger_name":"beinet.cn.dal.AppInfoMapper.findByAppKey","thread_name":"http-nio-8881-exec-1","level":"DEBUG","tags":["MYBATIS"]

以上是关于SpringBoot项目日志输出为JSON格式,方便Elastic采集的主要内容,如果未能解决你的问题,请参考以下文章

SpringBoot项目日志输出为JSON格式,方便Elastic采集

python日志输出的内容修改为json格式

24生产预警项目平台之Tomcat的支持log4j,日志输出为json格式

.net6 asp:在容器内运行时,默认输出日志格式为 json

Just Do IT,你的SpringBoot日志输出格式,由你来定!

Just Do IT,你的SpringBoot日志输出格式,由你来定!