liunx 使用crontab定时任务+shell脚本删除tomcat日志elasticsearch日志索引

Posted 2021-03-07 oldboyooxx

由于过多的日志很占用磁盘空间，今天经理让我写一个脚本删除多余的日志、和es索引

定时删除试根据crontab+shell脚本实现的

 

crontab配置目录

/var/spool/cron/

该目录下存放的是每个用户（包括root）的crontab任务，文件名以用户名命名

/etc/cron.d/

这个目录用来存放任何要执行的crontab文件或脚本。

crontab操作步骤

Step-One : 编辑任务脚本【分目录存放】【ex: backup.sh】
Step-Two : 编辑定时文件【命名规则:backup.cron】
Step-Three : crontab命令添加到系统crontab backup.cron
Step-Four : 查看crontab列表 crontab -l

 

Tomcat日志删除脚本

主要是shell脚本，第一次接触，写不好见谅（说明一下，公司多项目，所以用循环的方式删除）

#!/bin/bash
tomcat1="***-apiservice_beta"
tomcat2="***-gateway_beta"
tomcat3="***-mobile_beta"
tomcat4="***-mobile_preview"
tomcat5="***-portal_beta"
tomcat6="***-portal_preview"
tomcat7="***-portal-remote-freemarker_beta"
tomcat8="***-scheduler_beta"

#设定删除多少天之前的日志
day=30


for tomcat  in $tomcat1 $tomcat2 $tomcat3 $tomcat4 $tomcat5 $tomcat6 $tomcat7 $tomcat8
do
    find /opt/tomcat-web-apps/$tomcat/logs -name "host-manager.*" -mtime +$day -exec rm -rf {} ;
    find /opt/tomcat-web-apps/$tomcat/logs -name "localhost.*" -mtime +$day -exec rm -rf {} ;
    find /opt/tomcat-web-apps/$tomcat/logs -name "manager.*" -mtime +$day -exec rm -rf {} ;
    find /opt/tomcat-web-apps/$tomcat/logs -name "catalina.*" -mtime +$day -exec rm -rf {} ;
    find /opt/tomcat-web-apps/$tomcat/logs -name "localhost_access_log*" -mtime +60 -exec rm -rf {} ;
    find /opt/temp/$tomcat/logs -name "spring*" -mtime +$day -exec rm -rf {} ;
    find /opt/temp/$tomcat/logs/access -name "access*" -mtime +$day -exec rm -rf {} ;
done

elasticsearch日志删除脚本

具体脚本根据实际需求来

#!/bin/bash
elastic_url=127.0.0.1
elastic_port=9200
day=30
user_name=***
user_password=***
search_index1=**-log-gateway-service-main-beta
search_index2=**-log-gateway-service-access-beta
search_index3=**-log-portal-service-main-beta
search_index4=**-log-scheduler-service-main-beta
search_index5=**-log-service-main-beta
#删除早于day天的ES集群的索引

#获取当天时间
date2stamp () {
    date --utc --date "$1" +%s
}

#计算时间差
dateDiff (){
    case $1 in
        -s)   sec=1;      shift;;
        -m)   sec=60;     shift;;
        -h)   sec=3600;   shift;;
        -d)   sec=86400;  shift;;
        *)    sec=86400;;
    esac
    dte1=$(date2stamp $1)
    dte2=$(date2stamp $2)
    diffSec=$((dte2-dte1))
    if ((diffSec < 0)); then abs=-1; else abs=1; fi
    echo $((diffSec/sec*abs))
}

for search_index in $search_index1 $search_index2 $search_index3 $search_index4 $search_index5
do
    for index in $(curl -s "${elastic_url}:${elastic_port}/_cat/indices?v" -u ${user_name}:${user_password} | grep -E "${search_index}-20[0-9][0-9]-[0-1][0-9]-[0-3][0-9]" | awk ‘{ print $3 }‘;)
    do
      #循环当前索引的日期
      date=$(echo ${index: -$day} | sed ‘s/./-/g‘)
      #当天日期
      cond=$(date +%Y-%m-%d)
      #时间差
      diff=$(dateDiff -d $date $cond)
      echo -n "${index} (${diff})"
      if [ $diff -gt ${day} ]; then
        echo " yes DELETE"
        # curl -XDELETE "${elastic_url}:${elastic_port}/${index}?pretty"
      else
        echo " no DELETE"
      fi
    done
done

 

 

 

 

参考地址：

https://stackoverflow.com/questions/33430055/removing-old-indices-in-elasticsearch#answer-39746705

 https://www.jianshu.com/p/838db0269fd0