Tomcat+Memcached+Nginx实现session会话共享
Posted 包小哲
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了Tomcat+Memcached+Nginx实现session会话共享相关的知识，希望对你有一定的参考价值。
环境：
| ip | hostname | software version | OS | port |
| 192.168.1.120 | nginx-proxy-master.gz01 | nginx-1.8.1 | CentOS Linux release 7.4.1708 (Core) | 80 |
| 192.168.1.136 | tomcat-node01.gz01 | apache-tomcat-8.5.53 | CentOS Linux release 7.4.1708 (Core) | 8080 |
| 192.168.1.137 | tomcat-node02.gz01 | apache-tomcat-8.5.53 | CentOS Linux release 7.4.1708 (Core) | 8080 |
| 192.168.1.124 | mem-node01.gz01 | memcached-1.4.34 | CentOS Linux release 7.4.1708 (Core) | 11211 |
| 192.168.1.135 | mem-node02.gz01 | memcached-1.4.34 | CentOS Linux release 7.4.1708 (Core) | 11211 |
关闭防火墙和selinux，便于后面测试：
1 [root@mem-node01 ~]# systemctl stop firewalld
2 [root@mem-node02 ~]# systemctl stop firewalld
3 [root@tomcat-node01 ~]# systemctl stop firewalld
4 [root@tomcat-node02 ~]# systemctl stop firewalld
5 [root@nginx-proxy-master ~]# systemctl stop firewalld
1 [root@mem-node01 ~]# setenforce 0
2 [root@mem-node02 ~]# setenforce 0
3 [root@tomcat-node01 ~]# setenforce 0
4 [root@tomcat-node02 ~]# setenforce 0
5 [root@nginx-proxy-master ~]# setenforce 0
主机192.168.1.136 和192.168.1.137安装jdk和tomcat：
1 [root@tomcat-node01 tools]# ll
2 总用量 195372
3 -rw-r--r--. 1 root root 10300600 3月 25 21:42 apache-tomcat-8.5.53.tar.gz
4 -rw-r--r--. 1 root root 189756259 3月 25 22:08 jdk-8u161-linux-x64.tar.gz
5 [root@tomcat-node01 tools]# tar -zxvf jdk-8u161-linux-x64.tar.gz
6 [root@tomcat-node01 tools]# cp -r jdk1.8.0_161 /usr/local/jdk1.8.0_161
7 [root@tomcat-node01 tools]# ln -s /usr/local/jdk1.8.0_161/ /usr/local/jdk
8 [root@tomcat-node01 tools]# vim /etc/profile
9 JAVA_HOME=/usr/local/jdk/
10 CLASSPATH=$CLASSPATH:$JAVA_HOME/lib
11 PATH=$JAVA_HOME/bin:$PATH
12 [root@tomcat-node01 tools]# source /etc/profile
13 [root@tomcat-node01 tools]# java -version
14 java version "1.8.0_161"
15 Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
16 Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)
17
18 [root@tomcat-node01 tools]# tar -zxvf apache-tomcat-8.5.53.tar.gz
19 [root@tomcat-node01 tools]# mv apache-tomcat-8.5.53 /usr/local/
20 [root@tomcat-node01 tools]# ln -s /usr/local/apache-tomcat-8.5.53/ /usr/local/tomcat
21 [root@tomcat-node01 tools]# vim /etc/profile
22 CATALINA_HOME=/usr/local/tomcat/
23 PATH=$JAVA_HOME/bin:/usr/local/tomcat/bin/:$PATH
24 [root@tomcat-node01 tools]# source /etc/profile
25 [root@tomcat-node01 tools]# /usr/local/tomcat/bin/startup.sh
26 Using CATALINA_BASE: /usr/local/tomcat
27 Using CATALINA_HOME: /usr/local/tomcat
28 Using CATALINA_TMPDIR: /usr/local/tomcat/temp
29 Using JRE_HOME: /usr/local/jdk
30 Using CLASSPATH: /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
31 Tomcat started.
32 [root@tomcat-node01 tools]# netstat -lnt
33 Active Internet connections (only servers)
34 Proto Recv-Q Send-Q Local Address Foreign Address State
35 tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
36 tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN
37 tcp6 0 0 :::8080 :::* LISTEN
38 tcp6 0 0 :::22 :::* LISTEN
39 tcp6 0 0 ::1:25 :::* LISTEN
40 tcp6 0 0 127.0.0.1:8005 :::* LISTEN
41 [root@tomcat-node01 tools]#
42
43 tomcat-node02 也是一样操作
设置一下tomcat测试主页
1 [root@tomcat-node01 ROOT]# pwd
2 /usr/local/tomcat/webapps/ROOT
3 [root@tomcat-node01 ROOT]# cat index.jsp
4 <%@ page language="java" %>
5 <%@ page import="java.util.*" %>
6 <html>
7 <head>
8 <title>>Tomcat 192.168.1.136:8080 </title>
9 </head>
10 <body>
11
12 <h1><font color="blue">Tomcat 192.168.1.136:8080 </h1>
13 <table align="centre" border="1">
14 <tr>
15 <td>Session ID</td>
16 <% session.setAttribute("abc","abc"); %>
17 <td><%= session.getId() %></td>
18 </tr>
19 <tr>
20 <td>Created on</td>
21 <td><%= session.getCreationTime() %></td>
22 </tr>
23 </table>
24 </body>
25 </html>
26
27
28 [root@tomcat-node02 ROOT]# pwd
29 /usr/local/tomcat/webapps/ROOT
30 [root@tomcat-node02 ROOT]# cat index.jsp
31 <%@ page language="java" %>
32 <%@ page import="java.util.*" %>
33 <html>
34 <head>
35 <title>>Tomcat 192.168.1.137:8080 </title>
36 </head>
37 <body>
38
39 <h1><font color="blue">Tomcat 192.168.1.137:8080 </h1>
40 <table align="centre" border="1">
41 <tr>
42 <td>Session ID</td>
43 <% session.setAttribute("abc","abc"); %>
44 <td><%= session.getId() %></td>
45 </tr>
46 <tr>
47 <td>Created on</td>
48 <td><%= session.getCreationTime() %></td>
49 </tr>
50 </table>
51 </body>
52 </html>
nginx-proxy-master.gz01 192.168.1.120 安装nginx
1 安装依赖
2 [root@nginx-proxy-master ~]# rpm -ivh http://mirrors.aliyun.com/epel/epel-release-latest-7.noarch.rpm
3 Retrieving http://mirrors.aliyun.com/epel/epel-release-latest-7.noarch.rpm
4 warning: /var/tmp/rpm-tmp.K0JmFW: Header V3 RSA/SHA256 Signature, key ID 352c64e5: NOKEY
5 Preparing... ################################# [100%]
6 Updating / installing...
7 1:epel-release-7-12 ################################# [100%]
8 [root@nginx-proxy-master ~]# yum install -y gcc glibc gcc-c++ pcre-devel openssl-devel
1 [root@nginx-proxy-master tools]# pwd
2 /usr/local/tools
3 [root@nginx-proxy-master tools]# tar -zxvf nginx-1.8.1.tar.gz
4 [root@nginx-proxy-master tools]# cd nginx-1.8.1
5 [root@nginx-proxy-master nginx-1.8.1]# ./configure --prefix=/usr/local/nginx-1.8.1 \\
6 --with-http_ssl_module \\
7 --with-http_stub_status_module --with-file-aio
8 [root@nginx-proxy-master nginx-1.8.1]# make
9 [root@nginx-proxy-master nginx-1.8.1]# make install
10 [root@nginx-proxy-master nginx-1.8.1]# ln -s /usr/local/nginx-1.8.1/ /usr/local/nginx
11 [root@nginx-proxy-master nginx-1.8.1]# /usr/local/nginx/sbin/nginx
12 [root@nginx-proxy-master nginx-1.8.1]# netstat -nltp|grep 80
13 tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 12495/nginx: master
配置nginx
1 http {
2 include mime.types;
3 default_type application/octet-stream;
4 sendfile on;
5 upstream tomcat{
6 server 192.168.1.137:8080 weight=1;
7 server 192.168.1.136:8080 weight=2;
8 }
9
10 server {
11 listen 80;
12 server_name localhost;
13
14 location / {
15 proxy_pass http://tomcat;
16 }
17
18 location ~* \\.(jsp|do)$ {
19 proxy_pass http://tomcat;
20 proxy_set_header Host $host;
21 }
1 [root@nginx-proxy-master conf]# /usr/local/nginx/sbin/nginx -t
2 nginx: the configuration file /usr/local/nginx-1.8.1/conf/nginx.conf syntax is ok
3 nginx: configuration file /usr/local/nginx-1.8.1/conf/nginx.conf test is successful
4 [root@nginx-proxy-master conf]# /usr/local/nginx/sbin/nginx -s reload
访问192.168.1.120，发现访问请求结果会负载到192.168.1.136和192.168.1.137的tomcat上了。
在配置memcached-session-manager会话共享之前,访问nginx主机的请求会轮询负载到tomcat-node1和tomcat-node2两个节点上,并且session id会随着页面的刷新而改变,即此时还没有实现session会话共享
安装Memcached：(两个节点部署一样)
1 [root@mem-node02 tools]# yum -y install libevent libevent-devel
2 [root@mem-node02 tools]# tar -zxvf memcached-1.4.34.tar.gz
3 [root@mem-node02 tools]# cd memcached-1.4.34
4 [root@mem-node02 memcached-1.4.34]# ./configure --prefix=/usr/local/memcached
5 [root@mem-node02 memcached-1.4.34]# make
6 [root@mem-node02 memcached-1.4.34]# make install
7 [root@mem-node02 memcached-1.4.34]# /usr/local/memcached/bin/memcached -d -m 512 -u root -p 11211 -c 1024 -P /var/lib/memcached.11211pid
8 [root@mem-node02 memcached-1.4.34]#
9 [root@mem-node02 memcached-1.4.34]# ps -ef|grep memcached
10 root 3105 1 0 17:41 ? 00:00:00 /usr/local/memcached/bin/memcached -d -m 512 -u root -p 11211 -c 1024 -P /var/lib/memcached.11211pid
11 root 3138 1234 0 17:51 pts/0 00:00:00 grep --color=auto memcached
12 [root@mem-node02 memcached-1.4.34]# lsof -i:11211
13 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
14 memcached 3105 root 26u IPv4 34290 0t0 TCP *:memcache (LISTEN)
15 memcached 3105 root 27u IPv6 34291 0t0 TCP *:memcache (LISTEN)
16 memcached 3105 root 28u IPv4 34294 0t0 UDP *:memcache
17 memcached 3105 root 29u IPv4 34294 0t0 UDP *:memcache
18 memcached 3105 root 30u IPv4 34294 0t0 UDP *:memcache
19 memcached 3105 root 31u IPv4 34294 0t0 UDP *:memcache
20 memcached 3105 root 32u IPv6 34295 0t0 UDP *:memcache
21 memcached 3105 root 33u IPv6 34295 0t0 UDP *:memcache
22 memcached 3105 root 34u IPv6 34295 0t0 UDP *:memcache
23 memcached 3105 root 35u IPv6 34295 0t0 UDP *:memcache
配置Tomcat，通过MSM实现共享session（192.168.1.136和192.168.1.137机器上操作）
1、把memcached的jar包上传到/usr/local/tomcat/lib/目录
[root@tomcat-node01 ROOT]# cd /usr/local/tomcat/lib/
[root@tomcat-node01 lib]# ll
4 -rw-rw-r-- 1 sys-noc sysnoc-work 53259 8月 27 2018 asm-5.2.jar
5 -rw-rw-r-- 1 sys-noc sysnoc-work 323740 8月 27 2018 kryo-4.0.0.jar
6 -rw-rw-r-- 1 sys-noc sysnoc-work 85217 8月 27 2018 kryo-serializers-0.38.jar
7 -rw-rw-r-- 1 sys-noc sysnoc-work 152401 8月 27 2018 memcached-session-manager-1.9.7.jar
8 -rw-rw-r-- 1 sys-noc sysnoc-work 10788 8月 27 2018 memcached-session-manager-tc8-1.9.7.jar
9 -rw-rw-r-- 1 sys-noc sysnoc-work 5711 8月 27 2018 minlog-1.3.0.jar
10 -rw-rw-r-- 1 sys-noc sysnoc-work 37160 8月 27 2018 msm-kryo-serializer-1.9.7.jar
11 -rw-rw-r-- 1 sys-noc sysnoc-work 51287 8月 27 2018 objenesis-2.4.jar
12 -rw-rw-r-- 1 sys-noc sysnoc-work 20883 8月 27 2018 reflectasm-1.11.3.jar
13 -rw-rw-r-- 1 sys-noc sysnoc-work 472838 8月 27 2018 spymemcached-2.12.2.jar
2、修改conf/context.xml文件:(No-Stick模式)多个tomcat实例时 需要选择Non-Sticky模式，即sticky="false"
1 [root@tomcat-node01 conf]# cp context.xml context.xml.bak
2 [root@tomcat-node01 conf]# vim context.xml
3
4 <Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
5 memcachedNodes="n1:192.168.1.124:11211,n2:192.168.1.135:11211"
6 lockingMode="auto"
7 sticky="false"
8 sessionBackupAsync="false"
9 sessionBackupTimeout= "1000"
10 copyCollectionsForSerialization="true"
11 requestUriIgnorePattern=".*\\.(ico|png|gif|jpg|css|js)$"
12 transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory"
13 />
1 [root@tomcat-node01 conf]# /usr/local/tomcat/bin/shutdown.sh
2 Using CATALINA_BASE: /usr/local/tomcat
3 Using CATALINA_HOME: /usr/local/tomcat
4 Using CATALINA_TMPDIR: /usr/local/tomcat/temp
5 Using JRE_HOME: /usr/local/jdk
6 Using CLASSPATH: /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
7 [root@tomcat-node01 conf]# /usr/local/tomcat/bin/startup.sh
8 Using CATALINA_BASE: /usr/local/tomcat
9 Using CATALINA_HOME: /usr/local/tomcat
10 Using CATALINA_TMPDIR: /usr/local/tomcat/temp
11 Using JRE_HOME: /usr/local/jdk
12 Using CLASSPATH: /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
13 Tomcat started.
14 [root@tomcat-node01 conf]#
MSM会话共享测试
1、访问nginx主机,按ctrl+F5强刷页面,发现session信息会变,但是sessionid不会改变!说明session实现了共享! 如下,表示当前sessionid保存到了n2这个memcached节点上了.
2、关闭tomcat-node1和tomcat-node2中的任意一个节点的tomcat服务,继续访问页面,发现前端从nginx负载过来的请求达到未关闭的tomcat节点上,sessionid都不会改变,任然在共享中!即可以实现tomcat故障转移
[root@tomcat-node01 conf]# /usr/local/tomcat/bin/shutdown.sh
[root@tomcat-node02 conf]# /usr/local/tomcat/bin/shutdown.sh
3、关闭mem-node2节点的memcached服务,继续访问页面,发现sessionid保存到了n1这个memcached节点上了,但是sessionid任然没有改变,说明session已共享. 也就是说,关闭memcached集群中的任意一个节点.访问页面,sessionid都不会改变.即可以实现memcached故障转移!
以上是关于Tomcat+Memcached+Nginx实现session会话共享的主要内容，如果未能解决你的问题，请参考以下文章
Nginx反向代理+Tomcat+memcached实现session server
Tomcat+Memcached+Nginx实现session会话共享