Centos6下nginx+keepalived构建高可用web集群
Posted
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了Centos6下nginx+keepalived构建高可用web集群相关的知识，希望对你有一定的参考价值。
1）拓扑描述：
2) nginx的安装准备
pcre:兼容的正则表达式，nginx也要支持伪静态
# yum -y install pcre pcre-devel # yum -y install openssl* # mkdir -p /application/nginx1.6.2 # ln -s /application/nginx1.6.2 /application/nginx
3) 安装nginx
# cd /usr/local/src # tar xf nginx-1.6.2.tar.gz # cd nginx-1.6.2 # useradd nginx -s /sbin/nologin -M # ./configure --user=nginx --group=nginx --prefix=/application/nginx1.6.2 --with-http_stub_status_module --with-http_ssl_module # echo $? 0 # make && make install
4) 启动nginx
检查语法： # /application/nginx1.6.2/sbin/nginx -t nginx: the configuration file /application/nginx1.6.2/conf/nginx.conf syntax is ok nginx: configuration file /application/nginx1.6.2/conf/nginx.conf test is successful 启动nginx： # /application/nginx/sbin/nginx 查看端口号： # lsof -i :80 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME nginx 14603 root 6u IPv4 29397 0t0 TCP *:http (LISTEN) nginx 14604 nginx 6u IPv4 29397 0t0 TCP *:http (LISTEN) # netstat -tunlp | grep nginx tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 14603/nginx 测试网页页面： # curl -I localhost HTTP/1.1 200 OK Server: nginx/1.6.2 Date: Tue, 20 Sep 2016 02:17:20 GMT Content-Type: text/html Content-Length: 612 Last-Modified: Tue, 20 Sep 2016 02:11:05 GMT Connection: keep-alive ETag: "57e09ab9-264" Accept-Ranges: bytes
5）配置nginx启动脚本
# vim /etc/init.d/nginx
#!/bin/sh
# chkconfig: 2345 85 15
# description:Nginx Server
# nginx的安装目录
NGINX_HOME=/application/nginx
# nginx的命令
NGINX_SBIN=$NGINX_HOME/sbin/nginx
# nginx的配置文件
NGINX_CONF=$NGINX_HOME/conf/nginx.conf
# nginx的pid
NGINX_PID=$NGINX_HOME/logs/nginx.pid
NGINX_NAME="Nginx"
. /etc/rc.d/init.d/functions
if [ ! -f $NGINX_SBIN ]
then
echo "$NGINX_NAME startup: $NGINX_SBIN not exists! "
exit
fi
start() {
$NGINX_SBIN -c $NGINX_CONF
ret=$?
if [ $ret -eq 0 ]; then
action $"Starting $NGINX_NAME: " /bin/true
else
action $"Starting $NGINX_NAME: " /bin/false
fi
}
stop() {
kill `cat $NGINX_PID`
ret=$?
if [ $ret -eq 0 ]; then
action $"Stopping $NGINX_NAME: " /bin/true
else
action $"Stopping $NGINX_NAME: " /bin/false
fi
}
restart() {
stop
start
}
check() {
$NGINX_SBIN -c $NGINX_CONF -t
}
reload() {
kill -HUP `cat $NGINX_PID` && echo "reload success!"
}
relog() {
kill -USR1 `cat $NGINX_PID` && echo "relog success!"
}
case "$1" in
start)
start
;;
stop)
stop
;;
restart)
restart
;;
check|chk)
check
;;
status)
status -p $NGINX_PID
;;
reload)
reload
;;
relog)
relog
;;
*)
echo $"Usage: $0 {start|stop|restart|reload|status|check|relog}"
exit 1
esac
# chmod +x /etc/init.d/nginx
# /etc/init.d/nginx start
# chkconfig --add nginx
# chkconfig nginx on6) 配置nginx的upstream功能（两台负载均衡器上做相同的配置）
# egrep -v ‘#‘ /application/nginx/conf/nginx.conf|grep -v ‘^$‘
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
include extra/upstream01.conf;
sendfile on;
keepalive_timeout 65;
server {
listen 80;
server_name localhost;
location / {
root html;
index index.html index.htm;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}
说明：注意include extra/upstream01.conf这个文件，是引用此文件(两台负载均衡器上做系统的nginx配置)
# mkdir -p /application/nginx/conf/extra/
# vim /application/nginx/conf/extra/upstream01.conf
upstream nginx.wanwan.com {
server 10.10.10.128:80 weight=5;
server 10.10.10.132:80 weight=5;
}
server {
listen80;
server_namenginx.wanwan.com;
location / {
proxy_pass http://nginx.wanwan.com;
}
}
# /etc/init.d/nginx restart
Stopping Nginx: [确定]
Starting Nginx: [确定]7）keepalived的安装
# cd /usr/local/src # wget http://www.keepalived.org/software/keepalived-1.1.19.tar.gz # ln -s /usr/src/kernels/2.6.32-573.el6.x86_64/ /usr/src/linux # ls -l /usr/src 总用量 244 drwxr-xr-x. 2 root root 4096 9月 23 2011 debug -rw-r--r-- 1 root root 241437 1月 28 2014 keepalived-1.1.19.tar.gz drwxr-xr-x. 3 root root 4096 7月 5 23:49 kernels lrwxrwxrwx 1 root root 39 8月 31 08:49 linux -> /usr/src/kernels/2.6.32-573.el6.x86_64/ # tar xf keepalived-1.1.19.tar.gz # cd keepalived-1.1.19 # ./configure # make && make install # cp /usr/local/etc/rc.d/init.d/keepalived /etc/init.d/ # cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/ # mkdir -p /etc/keepalived # cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/ # cp /usr/local/sbin/keepalived /usr/sbin/ # /etc/init.d/keepalived start 正在启动 keepalived： [确定] # ps -ef | grep keepalived root 18750 1 0 22:55 ? 00:00:00 keepalived -D root 18752 18750 0 22:55 ? 00:00:00 keepalived -D root 18753 18750 0 22:55 ? 00:00:00 keepalived -D root 18755 18664 0 22:55 pts/0 00:00:00 grep keepalived keepalived-master的配置文件/etc/keepalived/keepalived.conf [[email protected] extra]# cat /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { [email protected] } notification_email_from [email protected] smtp_server smtp.qq.com smtp_connect_timeout 30 router_id nginx_7 } vrrp_instance VI_231 { state MASTER interface eth0 virtual_router_id 231 priority 150 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 10.10.10.231/24 } } } keepalived-slave的配置文件/etc/keepalived/keepalived.conf [[email protected] ~]# cat /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { [email protected] } notification_email_from [email protected] smtp_server smtp.qq.com smtp_connect_timeout 30 router_id nginx_7 } vrrp_instance VI_231 { state BACKUP interface eth0 virtual_router_id 231 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 10.10.10.231/24 } } }
8) 测试keepalived的功能（VIP为10.10.10.231)
[[email protected] extra]# ip add list 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:d7:3e:f8 brd ff:ff:ff:ff:ff:ff inet 10.10.10.131/24 brd 10.10.10.255 scope global eth0 inet 10.10.10.231/24 scope global secondary eth0 inet6 fe80::20c:29ff:fed7:3ef8/64 scope link valid_lft forever preferred_lft forever [[email protected] ~]# ip add list 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:71:33:eb brd ff:ff:ff:ff:ff:ff inet 10.10.10.135/24 brd 10.10.10.255 scope global eth0 inet6 fe80::20c:29ff:fe71:33eb/64 scope link valid_lft forever preferred_lft forever 关闭主负载均衡上的keepalived功能 [[email protected] extra]# /etc/init.d/keepalived stop 停止 keepalived： [确定] [[email protected] extra]# ip add list 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:d7:3e:f8 brd ff:ff:ff:ff:ff:ff inet 10.10.10.131/24 brd 10.10.10.255 scope global eth0 inet6 fe80::20c:29ff:fed7:3ef8/64 scope link valid_lft forever preferred_lft forever [[email protected] ~]# ip add list 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:71:33:eb brd ff:ff:ff:ff:ff:ff inet 10.10.10.135/24 brd 10.10.10.255 scope global eth0 inet 10.10.10.231/24 scope global secondary eth0 inet6 fe80::20c:29ff:fe71:33eb/64 scope link valid_lft forever preferred_lft forever 由上，我们可以知道vip很快就进行了切换，那么我们恢复主负载均衡器上的keepalived功能： [[email protected] extra]# /etc/init.d/keepalived start 正在启动 keepalived： [确定] [[email protected] extra]# ip add list 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:d7:3e:f8 brd ff:ff:ff:ff:ff:ff inet 10.10.10.131/24 brd 10.10.10.255 scope global eth0 inet 10.10.10.231/24 scope global secondary eth0 inet6 fe80::20c:29ff:fed7:3ef8/64 scope link valid_lft forever preferred_lft forever [[email protected] ~]# ip add list 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:71:33:eb brd ff:ff:ff:ff:ff:ff inet 10.10.10.135/24 brd 10.10.10.255 scope global eth0 inet6 fe80::20c:29ff:fe71:33eb/64 scope link valid_lft forever preferred_lft forever 由上，我们发现当主负载均衡器恢复后，vip很快就切换过来了（因为主负载均衡器上的优先级更高）
9）测试nginx的反向代理功能
[[email protected] ~]# curl 10.10.10.128 mysql successful by oldboy ! [[email protected] ~]# curl 10.10.10.132 this is web02‘s website
然后我们在客户端打开nginx.wanwan.com
按F5刷新：
[[email protected] extra]# /etc/init.d/nginx stop Stopping Nginx: [确定] [[email protected] extra]# ip add list 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:d7:3e:f8 brd ff:ff:ff:ff:ff:ff inet 10.10.10.131/24 brd 10.10.10.255 scope global eth0 inet 10.10.10.231/24 scope global secondary eth0 inet6 fe80::20c:29ff:fed7:3ef8/64 scope link valid_lft forever preferred_lft forever [[email protected] extra]# /etc/init.d/keepalived stop 停止 keepalived：
由上可知，后端网页仍旧正常。
10）注意事项
a、注意关闭负载均衡器以及web后端服务器的iptables以及selinux功能
b、两台负载均衡器上关于nginx配置是一致的，keepalived有不同的优先级
本文出自 “冰冻vs西瓜” 博客，请务必保留此出处http://molewan.blog.51cto.com/287340/1869558
以上是关于Centos6下nginx+keepalived构建高可用web集群的主要内容，如果未能解决你的问题，请参考以下文章