nginx+keepalived 部署web高可用
Posted LiuJun2Son
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了nginx+keepalived 部署web高可用相关的知识，希望对你有一定的参考价值。
1.准备两台在同一个局域网的Linux服务器
主机器ip：12.16.119.213
备用机器ip：12.16.119.214
2.两台服务器先都搭建好nginx服务器
1.两台服务器都安装keepalived
yum install keepalived -y # 先安装好nginx后再安装 keepalived
keepalived 配置文件路径：/etc/keepalived/keepalived.conf
cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak
2.两台服务器都修改keepalived配置
keepalived配置文件路劲：/etc/keepalived/keepalived.conf
主机器:12.16.119.213
! Configuration File for keepalived
global_defs
notification_email
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id LVS_DEVEL
vrrp_skip_check_adv_addr
# vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
#VIP1
vrrp_instance VI_1
state MASTER #
interface ens160 #
virtual_router_id 75 #
priority 100 #
advert_int 1
authentication
auth_type PASS
auth_pass 1111
virtual_ipaddress
12.16.119.200 #
12.16.119.20 #
12.16.110.200 #
修改前可以备份默认的配置：cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak
备用机器:12.16.119.214
! Configuration File for keepalived
global_defs
notification_email
acassen@firewall.loc # 接收邮件的邮箱
failover@firewall.loc
sysadmin@firewall.loc
notification_email_from Alexandre.Cassen@firewall.loc # 发邮件的邮箱
smtp_server 192.168.200.1 # 发邮件的邮箱地址
smtp_connect_timeout 30
router_id LVS_DEVEL
vrrp_skip_check_adv_addr
# vrrp_strict # 存在导致虚拟ip ping 不通
vrrp_garp_interval 0
vrrp_gna_interval 0
#VIP1
vrrp_instance VI_1 # 实例名称主备保持一致
state BACKUP # 主备配置为 MASTER ---- BACKUP
interface ens160 # ifconfig查看自己的网卡名称修改为它
virtual_router_id 75 # vrid 路由标识符，主备保持一致 0 - 255， 默认51
priority 90 # 优先级值越大越高 MASTER 100----BACKUP 90
advert_int 1
authentication # 认证，默认即可，主备保持一致
auth_type PASS
auth_pass 1111
virtual_ipaddress # vip，主备一致，可以有多个虚拟vip
12.16.119.200 # 可以
12.16.119.20 # 可以
12.16.110.200 # 可以
其实还可以继续添加备用机器:12.16.119.xxx
! Configuration File for keepalived
global_defs
notification_email
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id LVS_DEVEL
vrrp_skip_check_adv_addr
# vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
#VIP1
vrrp_instance VI_1
state BACKUP #
interface ens160 #
virtual_router_id 75 #
priority 80 #
advert_int 1
authentication
auth_type PASS
auth_pass 1111
virtual_ipaddress
12.16.119.200 #
12.16.119.20 #
12.16.110.200 #
如果 ifconfig 指令不能用可以执行 yum install net-tools 安装
3.两台服务器都启动keepalived
此外主机器上的防火墙和selinux必须关闭，否则keepalived无法拉起nginx。
临时关闭防火墙：systemctl stop firewalld
临时关闭selinux：setenforce 0
1.先启动两台服务器的nginx
/usr/local/webserver/nginx/sbin/nginx # 启动nginx的命令
2.再启动两台服务器的 keepalived
service keepalived start
[root@fxjc log]# service keepalived start
Redirecting to /bin/systemctl start keepalived.service
[root@fxjc log]#
2.两台服务器启动后，用 service keepalived status 看到 keepalived 的状态是正常的。
service keepalived status
[root@fxjc log]# service keepalived status
Redirecting to /bin/systemctl status keepalived.service
● keepalived.service - LVS and VRRP High Availability Monitor
Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)
Active: active (running) since Thu 2020-05-07 16:53:58 CST; 18s ago
Process: 887 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
Main PID: 888 (keepalived)
CGroup: /system.slice/keepalived.service
├─888 /usr/sbin/keepalived -D
├─889 /usr/sbin/keepalived -D
└─890 /usr/sbin/keepalived -D
May 07 16:54:05 fxjc Keepalived_vrrp[890]: Sending gratuitous ARP on ens160 for 12.16.110.200
May 07 16:54:05 fxjc Keepalived_vrrp[890]: Sending gratuitous ARP on ens160 for 12.16.119.20
May 07 16:54:05 fxjc Keepalived_vrrp[890]: Sending gratuitous ARP on ens160 for 12.16.119.200
May 07 16:54:05 fxjc Keepalived_vrrp[890]: Sending gratuitous ARP on ens160 for 12.16.110.200
May 07 16:54:05 fxjc Keepalived_vrrp[890]: Sending gratuitous ARP on ens160 for 12.16.119.20
May 07 16:54:05 fxjc Keepalived_vrrp[890]: Sending gratuitous ARP on ens160 for 12.16.119.200
May 07 16:54:05 fxjc Keepalived_vrrp[890]: Sending gratuitous ARP on ens160 for 12.16.110.200
May 07 16:54:05 fxjc Keepalived_vrrp[890]: Sending gratuitous ARP on ens160 for 12.16.119.20
May 07 16:54:05 fxjc Keepalived_vrrp[890]: Sending gratuitous ARP on ens160 for 12.16.119.200
May 07 16:54:05 fxjc Keepalived_vrrp[890]: Sending gratuitous ARP on ens160 for 12.16.110.200
[root@fxjc log]#
3.查看启动的情况
ip addr
[root@fxjc log]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:50:56:8b:71:fa brd ff:ff:ff:ff:ff:ff
inet 12.16.119.213/16 brd 12.16.255.255 scope global noprefixroute ens160
valid_lft forever preferred_lft forever
inet 12.16.119.20/32 scope global ens160
valid_lft forever preferred_lft forever
inet 12.16.119.200/32 scope global ens160
valid_lft forever preferred_lft forever
inet 12.16.110.200/32 scope global ens160
valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:fe8b:71fa/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@fxjc log]#
[root@fxjc log]#
4.启动失败查看keepalived的日志文件
keepalived 的日志文件记录在/var/log/messages文件内
5.停止服务
[root@fxjc log]# service keepalived stop
Redirecting to /bin/systemctl stop keepalived.service
[root@fxjc log]#
4.测试高可用
浏览器输入下面的地址都可以访问
注意nginx里面提前准备了pro项目，项目中只有一个index.html的文件
http://12.16.119.20/pro/index.html
http://12.16.110.200/pro/index.html
http://12.16.119.200/pro/index.html
5.仅停止主机器的keepalived
1.在主机器中执行
[root@fxjc log]# service keepalived stop
[root@fxjc log]#
2.在主机中查看虚拟ip （没有）
[root@fxjc log]# service keepalived stop
Redirecting to /bin/systemctl stop keepalived.service
[root@fxjc log]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:50:56:8b:71:fa brd ff:ff:ff:ff:ff:ff
inet 12.16.119.213/16 brd 12.16.255.255 scope global noprefixroute ens160
valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:fe8b:71fa/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@fxjc log]#
3.查看备用机器的虚拟ip （有）
[root@ocr bin]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:50:56:8b:6c:ba brd ff:ff:ff:ff:ff:ff
inet 12.16.119.214/16 brd 12.16.255.255 scope global noprefixroute ens160
valid_lft forever preferred_lft forever
inet 12.16.119.20/32 scope global ens160
valid_lft forever preferred_lft forever
inet 12.16.119.200/32 scope global ens160
valid_lft forever preferred_lft forever
inet 12.16.110.200/32 scope global ens160
valid_lft forever preferred_lft forever
inet6 fe80::924c:47f0:3651:2820/64 scope link noprefixroute
valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:f4:98:50:1d brd ff:ff:ff:ff:ff:ff
inet 192.168.128.1/24 brd 192.168.128.255 scope global docker0
valid_lft forever preferred_lft forever
inet6 fe80::42:f4ff:fe98:501d/64 scope link
valid_lft forever preferred_lft forever
[root@ocr bin]#
4.刷新网页发现，展示的内容也是备用机器中的网站
5.如果主机器重新启动keepalived, 那么虚拟ip会自动指向主机器
6.仅停止主机器的nginx
1.在主机器中执行
[root@fxjc log]# /usr/local/webserver/nginx/sbin/nginx -s stop
2.在主机中查看虚拟ip （有）
[root@fxjc log]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:50:56:8b:71:fa brd ff:ff:ff:ff:ff:ff
inet 12.16.119.213/16 brd 12.16.255.255 scope global noprefixroute ens160
valid_lft forever preferred_lft forever
inet 12.16.119.20/32 scope global ens160
valid_lft forever preferred_lft forever
inet 12.16.119.200/32 scope global ens160
valid_lft forever preferred_lft forever
inet 12.16.110.200/32 scope global ens160
valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:fe8b:71fa/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@fxjc log]#
3.查看备用机器的虚拟ip （没有生成）
[root@ocr bin]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:50:56:8b:6c:ba brd ff:ff:ff:ff:ff:ff
inet 12.16.119.214/16 brd 12.16.255.255 scope global noprefixroute ens160
valid_lft forever preferred_lft forever
inet6 fe80::924c:47f0:3651:2820/64 scope link noprefixroute
valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:f4:98:50:1d brd ff:ff:ff:ff:ff:ff
inet 192.168.128.1/24 brd 192.168.128.255 scope global docker0
valid_lft forever preferred_lft forever
inet6 fe80::42:f4ff:fe98:501d/64 scope link
valid_lft forever preferred_lft forever
4.刷新网页发现，网页无法访问了
5.如果主机器重新启动nginx，网页回复正常
建议nginx和keepalived同生死
7.监听nginx服务器的进程
主机器：建议 nginx 和 keepalived 同生死，如果监听到nginx挂了，那么也要把keepalived停止
1.主机器的的 keepalived.conf
! Configuration File for keepalived
global_defs
notification_email
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id LVS_DEVEL
vrrp_skip_check_adv_addr
# vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
vrrp_script chk_nginx
script "/etc/keepalived/nginx_check.sh"
#每1秒检测一次nginx的运行状态
interval 1
#失败一次，将自己的优先级调整为-2
weigth -2
#VIP1
vrrp_instance VI_1
state MASTER #
interface ens160 #
virtual_router_id 75 #
priority 100 #
advert_int 1
authentication
auth_type PASS
auth_pass 1111
virtual_ipaddress
12.16.119.200 #
12.16.119.20 #
12.16.110.200 #
#nginx存活状态检测脚本
track_script
chk_nginx
2.其中调用的 /etc/keepalived/nginx_check.sh 脚本内容为：
注意授权： chmod a+x nginx_check.sh 给这个脚本设置所有用户执行权限，不然这个脚本会调用失败
#!/usr/bin/bash
# 下面获取指令的返回值，不能用“” 或者 ‘’ 号
A=`ps -C nginx -no-header |wc -l`
# 下面[] 前后必须要给个空格 -eq 是等于
if [ $A -eq 1 ]
then
echo 'nginx has stop, stop keepalived too'
pkill keepalived
else
echo 'nginx run'
fi
3.测试
启动主机器的nginx 和 keepalived
/usr/local/webserver/nginx/sbin/nginx
service keepalived start
停止nginx服务器
/usr/local/webserver/nginx/sbin/nginx -s stop
查看keepalived，发现keepalived也停止了
service keepalived status
7.发生故障邮箱通知
1.安装依赖linux发送邮件的依赖包
yum -y install perl-CPAN # 按钮perl命令工具
> sudo perl -MCPAN -e shell # 进入capn
capn > install Net::SMTP_auth # 安装Net::SMTP_auth
2.在主机器下新建 /etc/keepalived/send_mail.sh 脚本
注意授权： chmod a+x nginx_check.sh 给这个脚本设置所有用户执行权限，不然这个脚本会调用失败
#!/usr/bin/perl -w
use Net::SMTP_auth;
use strict;
#smtp服务器
my $mailhost = 'smtp.qq.com';
#发送邮件的邮箱
my $mailfrom = '1552752805@qq.com';
#接收邮件的邮箱
my @mailto = ('635324221@qq.com');
#邮件主题
my $subject = 'keepalived up on backup';
#邮件正文
my $text = "正文\\n nginx-1服务器宕机！！nginx-2变为master！！！";
#发送邮件的用户名
my $user = '1552752805@qq.com';
#发送邮件的邮箱授权的密码
my $passwd = 'rxrgeedpitcbhxeb';
&SendMail();
##############################
# Send notice mail
##############################
sub SendMail()
my $smtp = Net::SMTP_auth->new( $mailhost, Timeout => 120, Debug => 1 )
or die "Error.\\n";
$smtp->auth( 'LOGIN', $user, $passwd );
foreach my $mailto (@mailto)
$smtp->mail($mailfrom);
$smtp->to($mailto);
$smtp->data();
$smtp->datasend("To: $mailto\\n");
$smtp->datasend("From:$mailfrom\\n");
$smtp-以上是关于nginx+keepalived 部署web高可用的主要内容，如果未能解决你的问题，请参考以下文章
基于Nginx搭建Web服务器HA架构（实现高可用web服务）
基于Nginx搭建Web服务器HA架构（实现高可用web服务）