Java日志信息存库(logback篇)

Posted 2020-09-20

一、Logback简介

　　Logback是由log4j创始人设计的又一个开源日志组件。logback当前分成三个模块：logback-core,logback- classic和logback-access。logback-core是其它两个模块的基础模块。logback-classic是log4j的一个 改良版本。此外logback-classic完整实现SLF4J API使你可以很方便地更换成其它日志系统如log4j或JDK14 Logging。logback-access访问模块与Servlet容器集成提供通过Http来访问日志的功能。

二、编写背景

　　很不明白客户为什么要求将日志信息写入到数据库中去，还要提供个页面给系统使用人员查看相应日志。作为一个业务人员真的就能看懂系统日志上报错信息是啥意思么，个人深表怀疑。没办法，作为一枚屌丝程序猿，需求下来了只能硬着头皮去开发。

三、编写目的

　　只相信一句话:好记性不如烂笔头，何况我记性差到前一周写的代码到现在竟毫无印象的境地呢。

四、Java日志信息存库详细解决方案

1.开发环境说明

　　Eclipse+Tomcat6+JDK1.6+Oracle+logback1.1

2.Java日志存库实现方案

(1)使用logback组件默认的DBAppender类实现

　　最初需求下来的时候想着logback应该有自己的写数据库的解决办法，于是乎结合源码及度娘终究还是找到了。在logback-classic-1.1.3.jar的ch/qos/logback/classic/db/script/路径下找到了Oracle数据库对应的建表语句脚本oracle.sql，其建表语句如下所示:

-- Logback: the reliable, generic, fast and flexible logging framework.
-- Copyright (C) 1999-2010, QOS.ch. All rights reserved.
--
-- See http://logback.qos.ch/license.html for the applicable licensing 
-- conditions.


-- This SQL script creates the required tables by ch.qos.logback.classic.db.DBAppender
--
-- It is intended for Oracle 9i, 10g and 11g databases. Tested on version 9.2, 
-- 10g and 11g.

-- The following lines are useful in cleaning any previously existing tables 

--drop TRIGGER logging_event_id_seq_trig; 
--drop SEQUENCE logging_event_id_seq; 
--drop table logging_event_property; 
--drop table logging_event_exception; 
--drop table logging_event; 


CREATE SEQUENCE logging_event_id_seq MINVALUE 1 START WITH 1;

CREATE TABLE logging_event 
  (
    timestmp         NUMBER(20) NOT NULL,
    formatted_message  VARCHAR2(4000) NOT NULL,
    logger_name       VARCHAR(254) NOT NULL,
    level_string      VARCHAR(254) NOT NULL,
    thread_name       VARCHAR(254),
    reference_flag    SMALLINT,
    arg0              VARCHAR(254),
    arg1              VARCHAR(254),
    arg2              VARCHAR(254),
    arg3              VARCHAR(254),
    caller_filename   VARCHAR(254) NOT NULL,
    caller_class      VARCHAR(254) NOT NULL,
    caller_method     VARCHAR(254) NOT NULL,
    caller_line       CHAR(4) NOT NULL,
    event_id          NUMBER(10) PRIMARY KEY
  );


-- the / suffix may or may not be needed depending on your SQL Client
-- Some SQL Clients, e.g. SQuirrel SQL has trouble with the following
-- trigger creation command, while SQLPlus (the basic SQL Client which
-- ships with Oracle) has no trouble at all.

CREATE TRIGGER logging_event_id_seq_trig
  BEFORE INSERT ON logging_event
  FOR EACH ROW  
  BEGIN  
    SELECT logging_event_id_seq.NEXTVAL 
    INTO   :NEW.event_id 
    FROM   DUAL;  
  END;
/


CREATE TABLE logging_event_property
  (
    event_id          NUMBER(10) NOT NULL,
    mapped_key        VARCHAR2(254) NOT NULL,
    mapped_value      VARCHAR2(1024),
    PRIMARY KEY(event_id, mapped_key),
    FOREIGN KEY (event_id) REFERENCES logging_event(event_id)
  );
  
CREATE TABLE logging_event_exception
  (
    event_id         NUMBER(10) NOT NULL,
    i                SMALLINT NOT NULL,
    trace_line       VARCHAR2(254) NOT NULL,
    PRIMARY KEY(event_id, i),
    FOREIGN KEY (event_id) REFERENCES logging_event(event_id)
  );

　　该sql脚本共创建了3张表,一个序列和一个触发器。其中主要日志信息记录在logging_event中，触发器是在logging_event数据新增时，将创建的序列值赋值给logging_event表的event_id。

1)JDBC连接池方式

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE configuration>
<configuration>
    
    <!-- JDBC方式将日志信息存入数据库-->
    <appender name="DB" class="ch.qos.logback.classic.db.DBAppender">  
    <connectionSource class="ch.qos.logback.core.db.DriverManagerConnectionSource">
        <!--此处使用的是阿里的数据连接池,也换成常用的C3P0数据连接池-->
        <dataSource class="com.alibaba.druid.pool.DruidDataSource">  
        <driverClass>oracle.jdbc.driver.OracleDriver</driverClass>  
        <url>jdbc:oracle:thin:@127.0.0.1:1521:orcl</url>  
        <user>tiger</user>  
        <password>123456</password>  
        </dataSource>  
    </connectionSource>  
    </appender>  
    
    <!--在logger标签指定使用的Appender-->
    <logger name="dblog" level="info">
        <appender-ref ref="DB" />
    </logger>
  
    <root level="debug" >
    </root>
</configuration>
    

 

2) JNDI方式

A.Tomcat服务器安装目录/conf/server.xml配置JNDI信息

<Context debug="0" docBase="E:\prj_abic\src\trunk\fundats\ats-modules-webservice\target\ats-modules-webservice" path="/webservice" reloadable="true">
    <Resource auth="Container" driverClassName="oracle.jdbc.driver.OracleDriver" maxActive="30" maxIdle="30" name="jdbc/logging" password="123456" type="javax.sql.DataSource" url="jdbc:oracle:thin:@127.0.0.1:1521:orcl" username="tiger"/>
</Context>

B.Spring配置文件applicationContext.xml配置JNDI信息

<bean id="dataSource" class="org.springframework.jndi.JndiObjectFactoryBean">
    <property name="jndiName">                    
    <!--此处引用Tomcat服务器名称为"jdbc/logging"的JNDI,WebLogic服务器可不填前缀直接写jdbc/logging即可-->
    <value>java:comp/env/jdbc/logging</value>          
    </property>
</bean>

C.logback.xml文件配置JNDI信息

?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE configuration>
<configuration>
    
    <!-- JNDI方式将日志信息存入数据库-->
   <appender name="DB" class="ch.qos.logback.classic.db.DBAppender">
    <connectionSource class="ch.qos.logback.core.db.JNDIConnectionSource">
        <!--此处引用Tomcat服务器名称为"jdbc/logging"的JNDI,WebLogic服务器可不填前缀直接写jdbc/logging即可-->
        <jndiLocation>java:comp/env/jdbc/logging</jndiLocation>
    </connectionSource>
   </appender>
    
    <!--在logger标签指定使用的Appender-->
    <logger name="dblog" level="info">
        <appender-ref ref="DB" />
    </logger>
  
    <root level="debug" >
    </root>
</configuration>

　　注:对于JNDI的配置不熟悉的，可以去找度娘帮忙或参考JNDI官方文档，本文不做细究。

(2)使用自定义DBAppender类实现

　　当初按上述的配置轻轻松松实现了日志信息存库，但天有不测风云，完成后却被告知客户那边任务触发器不安全，禁用触发器。得知此消息后心里仿佛有千万只草泥马飞奔而过，没办法只有继续想着怎么去改造了。于是就只能去自定义个DBAppender了，既然是自定义就干脆把其他两张表直接删了，仅使用logging_event表，其表结构如下：

字段名	中文说明	类型	为空
TIMESTMP	记录时间	NUMBER(20)	N
FORMATTED_MESSAGE	格式化后的日志信息	CLOB	N
LOGGER_NAME	执行记录请求的logger	VARCHAR2(256)	N
LEVEL_STRING	日志级别	VARCHAR2(256)	N
THREAD_NAME	日志线程名	VARCHAR2(256)	Y
REFERENCE_FLAG	包含标识：1-MDC或上下文属性;2-异常;3-均包含	INTEGER	Y
ARG0	参数1	VARCHAR2(256)	Y
ARG1	参数2	VARCHAR2(256)	Y
ARG2	参数3	VARCHAR2(256)	Y
ARG3	参数4	VARCHAR2(256)	Y
CALLER_FILENAME	文件名	VARCHAR2(256)	N
CALLER_CLASS	类	VARCHAR2(256)	N
CALLER_METHOD	方法名	VARCHAR2(256)	N
CALLER_LINE	行号	VARCHAR2(256)	N
EVENT_ID	主键ID	NUMBER(10)	N

　　注：这个表结构与logback提供的默认表结构有细微差别，主要体现在字段类型上，由于FORMATTED_MESSAGE存储的是详细的日志信息，字段类型VARCHAR2(4000)无法存储大文本信息，所以直接改造成了CLOB类型。

1)自定义DBAppender类(ATSDBAppender.java)

package com.hundsun.fund.ats.core.system.loggingevent.dao;

import java.lang.reflect.Method;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.SQLException;
import java.util.HashMap;import java.util.Map;
import static ch.qos.logback.core.db.DBHelper.closeStatement;
import static ch.qos.logback.core.db.DBHelper.closeConnection;
import ch.qos.logback.classic.db.DBHelper;
import ch.qos.logback.classic.db.names.DBNameResolver;
import ch.qos.logback.classic.db.names.DefaultDBNameResolver;
import ch.qos.logback.classic.spi.CallerData;
import ch.qos.logback.classic.spi.ILoggingEvent;
import ch.qos.logback.core.db.DBAppenderBase;

public class ATSDBAppender extends DBAppenderBase<ILoggingEvent>{
      protected String insertSQL;
      protected static final Method GET_GENERATED_KEYS_METHOD;

      private DBNameResolver dbNameResolver;

      static final int TIMESTMP_INDEX = 1;
      static final int  FORMATTED_MESSAGE_INDEX  = 2;
      static final int  LOGGER_NAME_INDEX = 3;
      static final int  LEVEL_STRING_INDEX = 4;
      static final int  THREAD_NAME_INDEX = 5;
      static final int  REFERENCE_FLAG_INDEX = 6;
      static final int  ARG0_INDEX = 7;
      static final int  ARG1_INDEX = 8;
      static final int  ARG2_INDEX = 9;
      static final int  ARG3_INDEX = 10;
      static final int  CALLER_FILENAME_INDEX = 11;
      static final int  CALLER_CLASS_INDEX = 12;
      static final int  CALLER_METHOD_INDEX = 13;
      static final int  CALLER_LINE_INDEX = 14;
      static final int  EVENT_ID_INDEX  = 15;

      static final StackTraceElement EMPTY_CALLER_DATA = CallerData.naInstance();

      static {
          Method getGeneratedKeysMethod;
        try {
            getGeneratedKeysMethod = PreparedStatement.class.getMethod("getGeneratedKeys", (Class[]) null);
        } catch (Exception ex) {
            getGeneratedKeysMethod = null;
        }
        GET_GENERATED_KEYS_METHOD = getGeneratedKeysMethod;
      }

      public void setDbNameResolver(DBNameResolver dbNameResolver) {
        this.dbNameResolver = dbNameResolver;
      }

      @Override
      public void start() {
        if (dbNameResolver == null)
            dbNameResolver = new DefaultDBNameResolver();
        insertSQL = ATSSQLBuilder.buildInsertSQL(dbNameResolver);
        super.start();
      }
      
      public void append(ILoggingEvent eventObject) {
            Connection connection = null;
            PreparedStatement insertStatement = null;
            try {
              connection = connectionSource.getConnection();
              connection.setAutoCommit(false);

              insertStatement = connection.prepareStatement(getInsertSQL());

              synchronized (this) {
                  subAppend(eventObject, connection, insertStatement);
              }
              connection.commit();
            } catch (Throwable sqle) {
                addError("problem appending event", sqle);
            } finally {
                closeStatement(insertStatement);
                closeConnection(connection);
            }
      }
      
      @Override
      protected void subAppend(ILoggingEvent event, Connection connection,PreparedStatement insertStatement) throws Throwable {

          bindLoggingEventWithInsertStatement(insertStatement, event);
          bindLoggingEventArgumentsWithPreparedStatement(insertStatement, event.getArgumentArray());

          bindCallerDataWithPreparedStatement(insertStatement, event.getCallerData());

          int updateCount = insertStatement.executeUpdate();
          if (updateCount != 1) {
              addWarn("Failed to insert loggingEvent");
          }
      }

      protected void secondarySubAppend(ILoggingEvent event, 
              Connection connection,long eventId) throws Throwable {
      }

      void bindLoggingEventWithInsertStatement(PreparedStatement stmt,
              ILoggingEvent event) throws SQLException {
          stmt.setLong(TIMESTMP_INDEX, event.getTimeStamp());
          stmt.setString(FORMATTED_MESSAGE_INDEX, event.getFormattedMessage());
          stmt.setString(LOGGER_NAME_INDEX, event.getLoggerName());
          stmt.setString(LEVEL_STRING_INDEX, event.getLevel().toString());
          stmt.setString(THREAD_NAME_INDEX, event.getThreadName());
          stmt.setShort(REFERENCE_FLAG_INDEX, DBHelper.computeReferenceMask(event));
      }

      void bindLoggingEventArgumentsWithPreparedStatement(PreparedStatement stmt,
          Object[] argArray) throws SQLException {

          int arrayLen = argArray != null ? argArray.length : 0;

          for(int i = 0; i < arrayLen && i < 4; i++) {
              stmt.setString(ARG0_INDEX+i, asStringTruncatedTo254(argArray[i]));
          }
          if(arrayLen < 4) {
              for(int i = arrayLen; i < 4; i++) {
                  stmt.setString(ARG0_INDEX+i, null);
              }
          }
      }

      String asStringTruncatedTo254(Object o) {
         String s = null;
         if(o != null) {
             s= o.toString();
         }

        if(s == null) {
          return null;
        }
        if(s.length() <= 254) {
          return s;
        } else {
          return s.substring(0, 254);
        }
      }

      void bindCallerDataWithPreparedStatement(PreparedStatement stmt,
          StackTraceElement[] callerDataArray) throws SQLException {

        StackTraceElement caller = extractFirstCaller(callerDataArray);

        stmt.setString(CALLER_FILENAME_INDEX, caller.getFileName());
        stmt.setString(CALLER_CLASS_INDEX, caller.getClassName());
        stmt.setString(CALLER_METHOD_INDEX, caller.getMethodName());
        stmt.setString(CALLER_LINE_INDEX, Integer.toString(caller.getLineNumber()));
      }

      private StackTraceElement extractFirstCaller(StackTraceElement[] callerDataArray) {
        StackTraceElement caller = EMPTY_CALLER_DATA;
        if(hasAtLeastOneNonNullElement(callerDataArray))
          caller = callerDataArray[0];
        return caller;
      }

      private boolean hasAtLeastOneNonNullElement(StackTraceElement[] callerDataArray) {
        return callerDataArray != null && callerDataArray.length > 0 && callerDataArray[0] != null;
      }

      Map<String, String> mergePropertyMaps(ILoggingEvent event) {
          Map<String, String> mergedMap = new HashMap<String, String>();
        Map<String, String> loggerContextMap = event.getLoggerContextVO().getPropertyMap();
        Map<String, String> mdcMap = event.getMDCPropertyMap();
        if (loggerContextMap != null) {
            mergedMap.putAll(loggerContextMap);
        }
        if (mdcMap != null) {
            mergedMap.putAll(mdcMap);
        }

        return mergedMap;
      }

      @Override
      protected Method getGeneratedKeysMethod() {
        return GET_GENERATED_KEYS_METHOD;
      }

      @Override
      protected String getInsertSQL() {
        return insertSQL;
      }
}

2)自定义SQLBuilder类(ATSSQLBuilder.java)

package com.hundsun.fund.ats.core.system.loggingevent.dao;

import ch.qos.logback.classic.db.names.ColumnName;
import ch.qos.logback.classic.db.names.DBNameResolver;
import ch.qos.logback.classic.db.names.TableName;

public class ATSSQLBuilder {
      static String buildInsertSQL(DBNameResolver dbNameResolver) {
        StringBuilder sqlBuilder = new StringBuilder("INSERT INTO ");
        sqlBuilder.append(dbNameResolver.getTableName(TableName.LOGGING_EVENT)).append(" (");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.TIMESTMP)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.FORMATTED_MESSAGE)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.LOGGER_NAME)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.LEVEL_STRING)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.THREAD_NAME)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.REFERENCE_FLAG)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.ARG0)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.ARG1)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.ARG2)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.ARG3)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.CALLER_FILENAME)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.CALLER_CLASS)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.CALLER_METHOD)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.CALLER_LINE)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.EVENT_ID)).append(") ");
        sqlBuilder.append("VALUES (?, ?, ? ,?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,LOGGING_EVENT_ID_SEQ.nextval)");
        return sqlBuilder.toString();
    }
}

　　注：类中LOGGING_EVENT_ID_SEQ序列需自行创建，logback默认建表语句中有该序列的创建语句，直接拿来使用即可。

3)logback.xml配置

　　此时JDBC与JNDI方式配置只需要将原name为DB的appender标签class属性的值指向自定义DBAppender即可,其他配置不变，下述代码为JDBC配置示例:

<!-- JDBC方式将日志信息存入数据库-->
<appender name="DB" class="com.hundsun.fund.ats.core.system.loggingevent.dao.ATSDBAppender">  
    <connectionSource class="ch.qos.logback.core.db.DriverManagerConnectionSource">  
    <dataSource class="com.alibaba.druid.pool.DruidDataSource">  
        <driverClass>oracle.jdbc.driver.OracleDriver</driverClass>  
        <url>jdbc:oracle:thin:@127.0.0.1:1521:orcl</url>  
        <user>tiger</user>  
        <password>123456</password>  
    </dataSource>  
    </connectionSource>  
</appender>

(3)测试类的编写

package com.hundsun.fund.ats.modules.server.test;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class LogbackTest {
    public static void main(String[] args) {
        //dblog为logback.xml中logger标签name属性的值
        Logger logger= LoggerFactory.getLogger("dblog");
        logger.debug("DEBUG级别信息");
        logger.warn("WARN级别信息");
        logger.info("INFO级别信息");
        logger.error("ERROR级别信息");
    }
}

五、总结

　　文中涉及到的Logback日志信息存库的处理只是在源代码的基础上做了点小小的改动而已，并非完整地介绍该组件的功能。想要全面学习Logback日志组件，请参考官方提供的源码和相应的API帮助文档。