服务器日志 之 一键部署log服务器
Posted
tags:
篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了服务器日志 之 一键部署log服务器相关的知识,希望对你有一定的参考价值。
1 概述
服务器上的日志,处理记录在本机上,也可以搭建专门收集log的服务器,方便分析。本文将介绍如何启用rsyslog的TCP和UDP的514端口进行网络日志收集,并通过rsyslog-mysql这个工具将日志记录到mysql数据库中, 再通过loganalyzer工具将日志用页面的形式展示出来
最后,本文将附上一键化部署安装log服务器的脚本
2 配置rsyslog成为日志服务器
要使得rsyslog成功日志服务器,从而收集其他主机的日志,要将TCP和UDP的514端口打开,通过修改配置文件/etc/rsyslog实现
配置如下
vim /etc/rsyslog #### MODULES #### # Provides UDP syslog reception $ModLoadim udp $UDPServerRun 514 # Provides TCP syslog reception $ModLoadim tcp $InputTCPServerRun 514
例子
设置一台机器为serverlog,专门用来收集日志,其他机器为客户端,不记录日志
假设log server为172.18.50.75这台,客户端上定义一个facility为local2,配置如下
服务器端
vim /etc/rsyslog.conf $ModLoad imudp#启用imudp模块 $UDPServerRun 514#开启UDP 514端口用来收集日志 $ModLoad imtcp #启用imtcp模块 $InputTCPServerRun 514 #开启TCP514端口用来收集日志 local2.* /var/serverlog/6Alocal2.log
#定义facility为local2的所有基本的日志都记录到/var/serverlog/6Alocal2.log,注意,这里6Alocal2.log这个log文件可以不用创建,只需重启rsyslog的服务,下次当满足条件的日志要记录的时候,系统就会自动生成。注意如果没有生成,可能是服务器端没有重启服务导致。
重启服务
service rsyslog restart
客户端
vim /etc/rsyslog.conf local2.* @172.18.50.75
#表示将facility为2的log记录到172.18.50.75这台机器上,其中,@表示UDP协议,@@表示UDP协议
重启服务
service rsyslog restart
3 rsyslog-mysql将日志记录于MySQL中
安装rsyslog-mysql(epel源中),将日志记录到mysql数据库中,安装过程会有脚本mysql-createDB.sql生成,需要在mysql服务器上执行该脚本。生成相关的库和表
.(1)准备MySQLServer
.(2)在rsyslog服务器上安装mysql模块相关的程序包(epel源中)
yum install rsyslog-mysql
.(3)为rsyslog创建数据库及表;安装rsyslog-mysql(epel源中),会有脚本/usr/share/doc/rsyslog-8.24.0/mysql-createDB.sql生成,将这个脚本导入到数据库中,用来生成表和数据库,
mysql -uUSERNAME -hHOST -pPASSWORD D < /usr/share/doc/rsyslog-7.4.7/mysql-createDB.sql
#执行数据库的脚本,表示用数据库账号直接执行mysql-createDB.sql这个脚本,注意用<小于号表示将mysql-createDB.sql这个脚本导入数据库
.(4)在mysqlserver上授权rsyslog能连接至当前服务器
mysql> GRANT ALL ON Syslog.* TO ‘USER‘@‘HOST‘ IDENTIFIED BY ‘PASSWORD‘;
.(5)配置rsyslog,将日志保存到mysql中
#### MODULES #### $ModLoadom mysql #### RULES #### facility.priority :ommysql:DBHOST,DBNAME,DBUSER, PASSWORD
:ommysql:这个是模块名,将来local2这个模式的日志将会安装以下的定义将日志记录到106的服务器上
4 通过loganalyzer展示数据库中的日志
loganalyzer基于php开发的,可以读取数据库的内容,以报表形式展现出来
.(1)在rsyslog服务器上准备amp或nmp组合
yum install httpd php php-mysql php-gd
php-mysql连接数据库用的
php-gd画图用的,epel源中
.(2)安装LogAnalyzer
tar xf loganalyzer-4.1.5.tar.gz
cp -a loganalyzer-4.1.5/src /var/www/html/log
#只拷贝该软件loganalyzer-4.1.5下src的目录就可以了
cd /var/www/html/log touch config.php chmod 666 config.php
#这个config.php文件只需要创建并赋予写的权限即可,不需要添加内容,该配置文件的内容将在重启服务后,在网页进行配置的时候写入。
.(3)配置loganalyzer
#重启httpd服务
systemctl start httpd.service
输入http://websrv/log进行配置,写入的内容将入记录到config.php这个文件里,注意MySQL Native, Syslog Fields,Monitorware的选择
打开网页,点击下一步进行配置
Source Type选择MYSQL Native,出现出现的配置。以下的信息中,默认的信息有问题,如表的大小写,如果这里写错信息,需要更改config.php这个文件,不修改的话,可以直接删掉该配置文件,重新创建并写入。
.(4)安全加强
cd /var/www/html/loganalyzer chmod 644 config.php
#将config.php改成只读模式,防止被其他人修改了相关配置。这一步建议操作
5 一键安装脚本
脚本使用前提
注意要配置好本地yum源和epel源,其中epel源建议使用sohu的epel源
loganalyzer这个软件包由于是解压安装,所以要提前准备好,或者有可以下载该服务包的路径。脚本中作者已经提前将这个服务包放到了自制的http服务器中http://172.18.50.75上。
有两个脚本
脚本一,用于一键化安装log服务器
脚本二,用于修改客户端的/etc/rsyslog.conf这个配置文件,使得log直接记录到对应的log服务器端
脚本一:一键安装log服务器
#!/bin/bash # #***************************************************************************************** #Author: Sunny #Date: 2017-10-16 #FileName: auto_install_log_server.sh #version: 1.0 #Your change info: #Description: For auto install log server by rsyslog-mysql and LogAnalyzer #DOC URL: #Copyright(C): 2017 All rihts reserved #***************************************************************************************** os_version=`cat /etc/system-release | grep -o " [0-9]"| cut -d " " -f2` time=`date +%Y%m%d%H%M` ip=$(ifconfig | awk ‘/inet /{print $2}‘| awk -F : ‘{print $NF}‘| head -1) package=‘loganalyzer-4.1.5.tar.gz‘ [ -e /root/package/package."$time" ] || mkdir -p /root/package/package."$time"; echo "$package" | tr -s " " "\n" &>/root/package/package.file echo install_rsyslog_mysq(){ rpm -q rsyslog-mysql &>/dev/null || { yum -y install rsyslog-mysql &>/dev/null && echo "rsyslog-mysql is install complete" || { echo "rsyslog-mysql is not install,check yum source";exit; }; } read -p "Input your sql admin user(default:root): " mysqladmin mysqladmin=${mysqladmin:-root} read -p "Input your sql admin user password: " adminpass createdb=$(rpm -ql rsyslog-mysql | grep createDB.sql) mysql -u$mysqladmin -p$adminpass < $createdb /usr/bin/mysql -u$mysqladmin -p$adminpass <<EOF grant all on Syslog.* to [email protected]‘%‘ identified by ‘Pass123456‘; EOF } config_rsyslog(){ rpm -q rsyslog &>/dev/null || { yum -y install rsyslog &>/dev/null && echo "rsyslog is install complete" || { echo "rsyslog is not install,check yum source";exit; }; } cat >>/etc/rsyslog.conf <<EOF \$ModLoad imudp \$UDPServerRun 514 \$ModLoad imtcp \$InputTCPServerRun 514 \$ModLoad ommysql local2.* :ommysql:$ip,Syslog,logadmin,Pass123456 EOF echo "rsyslog has been complete config,you can test if facility local2 can be log now." echo "You can add facility.loglevel :ommysql:$ip,Syslog,logadmin,Pass123456 to /etc/rsyslog.conf to log more log in the log server" } install_LogAnalyzer(){ echo "Now install loganalyzer" tar xf /root/package/package."$time"/$package -C /usr/local/ cp -a /usr/local/loganalyzer-4.1.5/src /var/www/html/log touch /var/www/html/log/config.php chmod 666 /var/www/html/log/config.php echo -e "LogAnalyzer has been release,please run http://$ip/blog to config your log admin,defautl config is below\n\n DBServer = "$ip";\n DBName = ‘Syslog‘;\n DBUser = ‘logadmin‘;\n DBPassword = ‘Pass123456‘;\n DBTableName = ‘SystemEvents‘;\n " } restart_service(){ service rsyslog restart &>/dev/null && echo "rsyslog has been restart" || echo "Something wrong when restart rsyslog,please check" service httpd restart &>/dev/null && echo "httpd has been restart" || echo "Something wrong when restart httpd,please check" case $os_version in 6) service mysqld restart &>/dev/null && echo "mysql has been restart" || echo "Something wrong when restart mysql,please check" ;; 7) service mariadb restart &>/dev/null && echo "mysql has been restart" || echo "Something wrong when restart mysql,please check" ;; *) echo "Something wrong when restart mysql,please check" exit ;; esac } install_pack(){ rpm -q httpd &>/dev/null || { yum -y install httpd &>/dev/null && echo "httpd is install complete" || { echo "httpd is not install,check yum source";exit; }; } rpm -q php &>/dev/null || { yum -y install php &>/dev/null && echo "php is install complete" || { echo "php is not install,check yum source";exit; }; } rpm -q php-mysql &>/dev/null || { yum -y install php-mysql &>/dev/null && echo "php-mysql is install complete" || { echo "php-mysql is not install,check yum source";exit; }; } rpm -q php-gd &>/dev/null || { yum -y install php-gd &>/dev/null && echo "php-gd is install complete" || { echo "php-gd is not install,check yum source";exit; }; } } download_LogAnalyzer(){ echo "You have two ways to get packages you want:" echo "remote: You will download from remote server,default url is http://192.168.32.75/source" echo "local: You have already prepare package in the local host" echo [ -e /root/package ] || mkdir -p /root/package; read -p "Your package in l(local) or r(remote)( r or l ): " choice case $choice in r) read -p "Please input the url where you want to download package(default:http://172.18.50.75/source): " url url=${url:-http://172.18.50.75/source} wget -nv --spider $url 2>&1 | grep -o "200 OK" &>/dev/null || { echo "The url is wrong or could not be connect,the scirpt will exit,please check";exit; } echo "Now start to download pack,please wait a minute" cd /root/package/package."$time" while read pack; do [ -e /root/package/package."$time"/$pack ] || wget -q "$url/$pack" [ -e /root/package/package."$time"/$pack ] && echo "$pack had been success download !" || { echo "$pack did not been downloaded,it will exist,please check...";exit; } done</root/package/package.file; # rm -f /root/package/package.file; ;; l) read -p "Please input the package directory(eg: /root/mariadb ): " localdir echo "Now start to copy pack to /root/package/package."$time",please wait a minute" cd /root/package/package."$time" while read pack do [ -e /root/package/package."$time"/$pack ] || cp $localdir/$pack /root/package/package."$time" &>/dev/null; [ -e /root/package/package."$time"/$pack ] && echo "$pack had been success copy to /root/package/package.$time " || { echo "$pack did not copy to /root/package/package.$time,it will exist,please check...";exit; } done</root/package/package.file # rm -f /root/package/package.file; ;; *) echo "Your input is not r or l ,and it is wrong input,the script will exit,please check" exit ;; esac } echo echo "First of all,make your basic and epel source is ok,it is better sohu epel,you can run cmd yum repolist to check your yum source" echo read -p "Is your epel ok?,answer y to continue,other to check your epel: " isepel case $isepel in y) echo "Since your answer is y,I know your epel is ok,the script will continue..." ;; *) echo "For your answer is not y,it will exit,please check your epel" echo -e "Below is how to config sohu epel,you can write it to /etc/yum.repo.d/sunny.repo\n\n [sohu]\n name=sohu-source\n baseurl=http://mirrors.sohu.com/centos/\$releasever/os/\$basearch/\n gpgcheck=1\n enabled=0\n gpgkey=http://mirrors.sohu.com/centos/\$releasever/os/\$basearch/RPM-GPG-KEY-CentOS-\$releasever\n" echo exit ;; esac case $os_version in 6) if rpm -q mysql-server &>/dev/null;then echo "The mysql-server is already install before" else rpm -q mysql &>/dev/null || { yum -y install mysql &>/dev/null && echo "mysql is install complete" || { echo "mysql is not install,check yum source";exit; }; } rpm -q mysql-server &>/dev/null || { yum -y install mysql-server &>/dev/null && echo "mysql-server is install complete" || { echo "mysql is not install,check yum source";exit; }; } service mysqld restart &>/dev/null && echo "mysql has been restart" || echo "Something wrong when restart mysql,please check" /usr/bin/mysql_secure_installation; fi echo "Now install some relative package" install_pack install_rsyslog_mysq config_rsyslog download_LogAnalyzer install_LogAnalyzer ;; 7) if rpm -q mysql-server &>/dev/null;then echo "The mysql-server is already install before" else rpm -q mariadb &>/dev/null || { yum -y install mariadb &>/dev/null && echo "mysql is install complete" || { echo "mysql is not install,check yum source";exit; }; } rpm -q mariadb-server &>/dev/null || { yum -y install mariadb-server &>/dev/null && echo "mysql-server is install complete" || { echo "mysql-server is not install,check yum source";exit; }; } service mariadb restart &>/dev/null && echo "mysql has been restart" || echo "Something wrong when restart mysql,please check" /usr/bin/mysql_secure_installation fi echo "Now install some relative package" install_pack install_rsyslog_mysq config_rsyslog download_LogAnalyzer install_LogAnalyzer ;; *) echo "Your system is not centos6 or 7,please check" exit ;; esac echo echo "All config is done now,Now restart service" restart_service echo "If all service is restart ok,you can test now,otherwise,you just to solve the restart problem,the test" echo "test url is http://$ip/blog"
脚本二:一键配置客户端/etc/rsyslog.conf
#!/bin/bash # #****************************************************************************** #Author: Sunny #Date: 2017-10-15 #FileName: auto_set_rsyslog_conf.sh #version: 1.0 #Your change info: #Description: For auto set rsylog_conf in client #DOC URL: #Copyright(C): 2017 All rihts reserved #***************************************************************************** time=`date +%Y%m%d%H%M` os_version=`cat /etc/system-release | grep -o " [0-9]"| cut -d " " -f2` mv /etc/rsyslog.conf /etc/rsyslog.conf.$time.bak read -p "Please input your log server ip(default:172.18.50.75): " ip ip=${ip:-172.18.50.75} echo ip is $ip case $os_version in 6) cat >/etc/rsyslog.conf<<eof \$ModLoad imuxsock # provides support for local system logging (e.g. via logger command) \$ModLoad imklog # provides kernel logging support (previously done by rklogd) \$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat \$IncludeConfig /etc/rsyslog.d/*.conf *.info;mail.none;authpriv.none;cron.none @$ip authpriv.* @$ip mail.* @$ip cron.* @$ip *.emerg * uucp,news.crit @$ip local7.* @$ip eof ;; 7) cat >/etc/rsyslog.conf<<eof \$ModLoad imuxsock # provides support for local system logging (e.g. via logger command) \$ModLoad imjournal # provides access to the systemd journal \$WorkDirectory /var/lib/rsyslog \$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat \$IncludeConfig /etc/rsyslog.d/*.conf \$OmitLocalLogging on \$IMJournalStateFile imjournal.state *.info;mail.none;authpriv.none;cron.none @$ip authpriv.* @$ip mail.* @$ip cron.* @$ip *.emerg :omusrmsg:* uucp,news.crit @$ip local7.* @$ip eof ;; *) echo "The host is not centos6 or 7,it will exit now" mv /etc/rsyslog.conf.$time.bak /etc/rsyslog.conf exit ;; esac service rsyslog restart && echo "rsyslog has been restart" || echo "something wrong when restart rsyslog,please check"
6 总结
本文实现了一键安装安装log服务器,关于日志级别的定义,以及日志的存储规则,可以参见博客服务器日志 之 rsyslog和logrotate 概念介绍 http://ghbsunny.blog.51cto.com/7759574/1972977进行按需配置。
本文出自 “阳光运维” 博客,请务必保留此出处http://ghbsunny.blog.51cto.com/7759574/1973012
以上是关于服务器日志 之 一键部署log服务器的主要内容,如果未能解决你的问题,请参考以下文章
dockerdocker部署spring boot服务,但是docker logs查看容器输出控制台日志,没有日志打印,日志未打印,docker logs不打印容器日志