通过crontab监控SparkStreaming任务运行状态

Posted 2020-11-23 wuning

1.问题

最近发现SparkStreaming提交的job经常在半夜挂掉，于是写了个定时任务监控SparkStreaming的运行状态，保证其不挂掉

2.shell脚本

touch /opt/module/jobs/monitorlog.txt
vim /opt/module/jobs/monitor.sh

#!/bin/bash

#在linux中查找你所运行的spark任务中  任务名称为WBStreamingClusterDriver的任务有没有,如果有则返回值是1
job_status=$(yarn application -list| awk '{print $2}'  | grep KeyBehaviorsQl  | wc -l)

if [ $job_status = 0 ];then

    echo $(date "+%Y-%m-%d %H:%M:%S")  'SparkStreamingTestis stop'  >> /opt/module/jobs/monitorlog.txt

    nohup spark2-submit --master yarn --deploy-mode cluster --driver-memory 3g --num-executors 70 --executor-cores 2 --executor-memory 3g  --class com.eighteen.sparkstreaming.SparkStreamingTest/opt/module/jobs/SparkStreaming-1.0-SNAPSHOT-jar-with-dependencies.jar

    echo $(date "+%Y-%m-%d %H:%M:%S")  "restart SparkStreamingTest success !!!"  >> /opt/module/jobs/monitorlog.txt

else

    echo $(date "+%Y-%m-%d %H:%M:%S")  'SparkStreamingTest is running !!' >> /opt/module/jobs/monitorlog.txt

fi

3.编写定时任务

*/3 * * * * /opt/module/jobs/monitor.sh

4.查看 monitorlog.txt