部署在tomcat下面的jenkins启动了多个实例，界面访问报错

Posted 2020-11-07

问题描述：

最近因为机房停电，机器上面的所有服务有出现重启，包括jenkins服务。jenkins在重启之后出现了不能访问，界面报错信息：

报错说启动了多个实例，点击下面的“Ignore this problem and keep using Jenkins anyway”按钮，可以恢复到原来正常的界面，但是这个报错，不久又会出现。

原因分析

因为jenkins是放在tomcat下面的，查看tomcat进程，发现确实只有一个，端口也没有什么异常：

[[email protected] bin]# ps -ef|grep tomcat
root     21780     1 43 15:15 pts/2    00:00:12 /usr/local/jdk/bin/java -Djava.util.logging.config.file=/usr/local/apache-tomcat-8.0.48/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dignore.endorsed.dirs= -classpath /usr/local/apache-tomcat-8.0.48/bin/bootstrap.jar:/usr/local/apache-tomcat-8.0.48/bin/tomcat-juli.jar -Dcatalina.base=/usr/local/apache-tomcat-8.0.48 -Dcatalina.home=/usr/local/apache-tomcat-8.0.48 -Djava.io.tmpdir=/usr/local/apache-tomcat-8.0.48/temp org.apache.catalina.startup.Bootstrap start
root     12245  7771  0 15:16 pts/2    00:00:00 grep --color=auto tomcat
[[email protected] bin]# 
[[email protected] bin]# 
[[email protected] bin]# 
[[email protected] bin]# 
[[email protected] bin]# netstat -tlunp|grep 28080
tcp        0      0 0.0.0.0:23000           0.0.0.0:*               LISTEN      28080/fdfs_storaged 
tcp6       6      0 :::28080                :::*                    LISTEN      12168/java          
[[email protected] bin]# 

根据https://stackoverflow.com/questions/44378841/jenkins-detecting-more-than-one-instance的提示：
? Check if another java process is running which runs jenkins e.g. in unix using top
? If you‘re using tomcat, search for a second instance of tomcat having the jenkins.war exploded somewhere
? Could be helpful to just have one version of java installed

查看jenkins，怀疑可能是哪里启动了多个jenkins进程，发现有两个一模一样的jenkins进程，只是进程号不一样：

[[email protected] home]# ps -ef|grep jenkins
samba    10835 10820  0 May18 ?        00:00:05 /bin/tini -- /usr/local/bin/jenkins.sh
samba    10854 10835  0 May18 ?        00:03:53 java -Duser.home=/var/jenkins_home -jar /usr/share/jenkins/jenkins.war
root     30593 21780  0 10:34 pts/2    00:00:00 sh -c { while [ -d ‘/root/.jenkins/workspace/[email protected]/durable-9c600ea7‘ -a \! -f ‘/root/.jenkins/workspins-result.txt‘ ]; do touch ‘/root/.jenkins/workspace/[email protected]/durable-9c600ea7/jenkins-log.txt‘; sleep 3; done } & jsc=durable-95de6aeeb4c40d371cff43dc ‘/root/.jenkins/workspace/[email protected]/durable-9c600ea7/script.sh‘ > ‘/root/.jenkins/workspace/[email protected]/durable-9c600ea7/jenkins-log.txt‘ 2>&1; echo $?tmp/durable-9c600ea7/jenkins-result.txt‘; wait  #这个进程后面的21780看，应该是之前tomcat正常启动的进程
*root     30596 30593  0 10:34 pts/2    00:00:00 sh -c { while [ -d ‘/root/.jenkins/workspace/[email protected]/durable-9c600ea7‘ -a \! -f ‘/root/.jenkins/workspins-result.txt‘ ]; do touch ‘/root/.jenkins/workspace/[email protected]/durable-9c600ea7/jenkins-log.txt‘; sleep 3; done } & jsc=durable-95de6aeeb4c40d371cff43dc ‘/root/.jenkins/workspace/[email protected]/durable-9c600ea7/script.sh‘ > ‘/root/.jenkins/workspace/[email protected]/durable-9c600ea7/jenkins-log.txt‘ 2>&1; echo $?tmp/durable-9c600ea7/jenkins-result.txt‘; wait  #这个进程有点异常
root     30597 30593  0 10:34 pts/2    00:00:00 /bin/sh -xe /root/.jenkins/workspace/[email protected]/durable-9c600ea7/script.sh
root     30600 30597 44 10:34 pts/2    00:00:13 /usr/local/jdk/bin/java -classpath /var/jenkins_home/apache-maven-3.5.2/boot/plexus-classworlds-2.5.2.jare/apache-maven-3.5.2/bin/m2.conf -Dmaven.home=/var/jenkins_home/apache-maven-3.5.2 -Dlibrary.jansi.path=/var/jenkins_home/apache-maven-3.5.2/lib/jansi-natory=/home/.jenkins/workspace/locman/locman-service-controller org.codehaus.plexus.classworlds.launcher.Launcher package -DskipTests
root     30702 30600  0 10:35 pts/2    00:00:00 /bin/sh -c cd /home/.jenkins/workspace/locman/locman-service-controller && /usr/local/jdk/jre/../bin/java.javac.JavacCompiler8324825440823874050arguments
root     31210  7771  0 10:35 pts/2    00:00:00 grep --color=auto jenkins
[[email protected] home]# 

原因说明：
因为jenkins是放在tomcat下面的，将jenkins.war解压到了/usr/local/apache-tomcat-8.0.48/webapps/ROOT目录下面，但是还放了一个jenkins.war在webapps下面忘记删掉。因此在启动tomcat的时候，就一起启动了webapps下面的jenkins.war和ROOT目录下面的jenkins包内容。所以就启动了两个jenkins实例。

jenkins.war放在webapps目录下面和解压到ROOT目录下面的区别：
ROOT目录下面默认是tomcat的管理程序，但是如果你把自己的jenkins发布到root下面的话，你可以不通过项目名直接访问你的项目，直接使用通过http://localhost:8080直接访问你的项目（下图端口号用的28080）：

如果放在webapps下面，访问的时候，就需要通过项目名称来访问http://localhost:8080/jenkins/（下图端口号用的28080）：

解决方法：

杀掉所有的jenkins和tomcat进程，然后将webapps下面的jenkins.war改个名字或者删掉，再重启tomcat

[[email protected] webapps]# mv jenkins.war  jenkins.war.bak
[[email protected] bin]#  
[[email protected] bin]# ./startup.sh 
[[email protected] webapps]# ps -ef|grep jenkins
root      3611  7771  0 10:54 pts/2    00:00:00 grep --color=auto jenkins
samba    10835 10820  0 May18 ?        00:00:05 /bin/tini -- /usr/local/bin/jenkins.sh
samba    10854 10835  0 May18 ?        00:03:54 java -Duser.home=/var/jenkins_home -jar /usr/share/jenkins/jenkins.war
[[email protected] webapps]# ps -ef|grep tomcat
root      1238     1 19 10:41 pts/2    00:02:33 /usr/local/jdk/bin/java -Djava.util.logging.config.file=/usr/local/apache-tomcat-8.0.48/conf/logging.proprg.apache.juli.ClassLoaderLogManager -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dignore.endorsed.dicat-8.0.48/bin/bootstrap.jar:/usr/local/apache-tomcat-8.0.48/bin/tomcat-juli.jar -Dcatalina.base=/usr/local/apache-tomcat-8.0.48 -Dcatalina.home=/usr/locpdir=/usr/local/apache-tomcat-8.0.48/temp org.apache.catalina.startup.Bootstrap start
root      3632  7771  0 10:54 pts/2    00:00:00 grep --color=auto tomcat

然后暂时没有出现这个错误了。

备注：webapps下面的jenkins.war一直都存在，在异常停电重启之前，没有发现报这个错。目前暂时还没出现上面的报错了，问题是否解决还需后续继续观察