Nginx反向代理-Keepalived做高可用
Posted
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了Nginx反向代理-Keepalived做高可用相关的知识，希望对你有一定的参考价值。
Keepalived-HA-主节点配置管理
1.软件安装
[root@ha1 nginx]# yum install -y keepalived
[root@ha1 nginx]# cd /etc/keepalived/
[root@ha1 keepalived]# mv keepalived.conf keepalived.conf.bak
2.主备配置-集群功能
[root@ha1 keepalived]# vim keepalived.conf
! Configuration File for keepalived
global_defs
notification_email
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1 #邮箱地址
smtp_connect_timeout 30
router_id keepalived_master # 真实路由器ID - 主备节点参数需要不同
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
vrrp_instance VI_1
state MASTER #节点状态
interface ens33 #节点keepalived接⼝IP地址
virtual_router_id 131 #虚拟路由器ID - - 主备节点参数需要相同
#尽可能⾃定义 1-255 ｜防⽌冲突
priority 150 #选举优先级 - Master节点优先级⾼
advert_int 1 # ⼼跳通告时间间隔 advertisement-internal - 1S |检测倍数3倍
authentication
auth_type PASS #开始认证
auth_pass Admin@123 #
virtual_ipaddress
192.168.40.113
3.启动与自启动
[root@ha1 keepalived]# systemctl start keepalived.service && systemctl enable keepalived.service
[root@ha1 keepalived]# systemctl status keepalived.service
4.查看网址信息
[root@ha1 keepalived]# ip add
2: ens33: mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:15:8f:0c brd ff:ff:ff:ff:ff:ff
inet 192.168.40.111/24 brd 192.168.40.255 scope global ens33 #本机⽹卡真实地址
valid_lft forever preferred_lft forever
inet 192.168.40.113/32 scope global ens33 # 集群虚拟IP地址 （主）
Keepalived-HA-备节点配置管理
1.软件安装
[root@ha2 nginx]# yum install -y keepalived
[root@ha2 nginx]# cd /etc/keepalived/
[root@ha2 keepalived]# mv keepalived.conf keepalived.conf.bak
2.文件配置
[root@ha2 keepalived]# vim keepalived.conf
! Configuration File for keepalived
global_defs
notification_email
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1 #邮箱地址
smtp_connect_timeout 30
router_id keepalived_backup # 真实路由器ID - 主备节点参数需要不同
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
vrrp_instance VI_1
state BACKUP #节点状态
interface ens33 #节点keepalived接⼝IP地址
virtual_router_id 131 #虚拟路由器ID - 主备节点参数需要相同
#尽可能⾃定义 1-255 ｜防⽌冲突
priority 90 #选举优先级 - Master节点优先级⾼
advert_int 1 # ⼼跳通告时间间隔 advertisement-internal - 1S |检测倍数3倍
authentication
auth_type PASS #开始认证
auth_pass Admin@123 #
virtual_ipaddress
192.168.40.113
3、启动与自启动
[root@ha2 keepalived]# systemctl start keepalived.service && systemctl enable keepalived.service
[root@ha2 keepalived]# systemctl status keepalived.service
4、查看网址信息
[root@ha2 keepalived]# ip add 2: ens33: mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:30:ed:19 brd ff:ff:ff:ff:ff:ff inet 192.168.40.112/24 brd 192.168.40.255 scope global ens33
Keepalived-HA-主备倒换测试概述
主备倒换测试
1.模拟主节点故障 - 观察，⽤户访问流量是否能正常切换到备份节点。 ----- Keepalived服务停⽌
2.恢复主节点 - 观察，⽤户访问流量是否能正常回切到主节点。 ----- Keepalived服务正常打开
Keepalived-HA-主备倒换测试操作
[root@ha1 keepalived]# systemctl stop keepalived.service
root@ha2 keepalived]# systemctl status keepalived.service
● keepalived.service - LVS and VRRP High Availability Monitor
Loaded: loaded (/usr/lib/systemd/system/keepalived.service; enabled; vendor preset: disabled)
Active: active (running) since Fri 2022-12-23 10:27:48 CST; 4h 20min ago
Main PID: 14663 (keepalived)
CGroup: /system.slice/keepalived.service
├─14663 /usr/sbin/keepalived -D
├─14664 /usr/sbin/keepalived -D
└─14665 /usr/sbin/keepalived -D
Dec 23 14:48:17 ha2 Keepalived_vrrp[14665]: VRRP_Instance(VI_1) Transition to MASTER STATE
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: VRRP_Instance(VI_1) Entering MASTER STATE #就是这个位置
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: VRRP_Instance(VI_1) setting protocol iptable drop rule
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: VRRP_Instance(VI_1) setting protocol VIPs.
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: Sending gratuitous ARP on ens33 for 192.168.40.113
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ens33 for 192.168.40.113
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: Sending gratuitous ARP on ens33 for 192.168.40.113
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: Sending gratuitous ARP on ens33 for 192.168.40.113
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: Sending gratuitous ARP on ens33 for 192.168.40.113
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: Sending gratuitous ARP on ens33 for 192.168.40.113
Keepalived-HA-Nginx-健康检测
[root@ha1 keepalived]# systemctl stop nginx
[root@ha1 keepalived]#
[root@ha1 keepalived]# ip add show ens192
2: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen
1000
link/ether 00:50:56:94:4a:c5 brd ff:ff:ff:ff:ff:ff
inet 192.168.40.111/24 brd 192.168.8.255 scope global ens192
valid_lft forever preferred_lft forever
inet 192.168.8.113/32 scope global ens192 #集群IP地址
valid_lft forever preferred_lft forever
# 当Nginx⽆法提供服务时，当前节点⾃动关闭keepalived
# 主节点Nginx⽆法提供服务时 ，keepalived⾃动关闭 - 不再发送⼼跳消息 - 备份节点如果nginx可
⽤切换keepalived主节点。
1、抓取个数
[root@ha1 keepalived]# netstat -ntlp | grep "nginx: master" | wc -l
1 #nginx端⼝监听成功
[root@ha2 keepalived]# netstat -ntlp | grep "nginx: master" | wc -l
0 #nginx端⼝监听失败
1.检测监听端⼝
2.监听没有开启 - 尝试拉活
3.如果⽆法拉活 - 关闭keepalived
2、自启脚本
cd /etc/keepalived
vim check_nginx_service_port.sh
#!/bin/bash
if [ "$(netstat -ntlp | grep "nginx: master" | wc -l)" == "0" ]
then
systemctl restart nginx
sleep 2
if [ "$(netstat -ntlp | grep "nginx: master" | wc -l)" == "0" ]
then
systemctl stop keepalive
fi
fi
[root@ha1 keepalived]# chmod +x check_nginx_service_port.sh
! Configuration File for keepalived
global_defs
notification_email
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1 #邮箱地址
smtp_connect_timeout 30
router_id keepalived_master # 真实路由器ID - 主备节点参数需要不同
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
vrrp_script keepalived_check_nginx_service_port
script "/etc/keepalived/check_nginx_service_port.sh"
interval 2 # 执⾏脚本时间间隔
vrrp_instance VI_1
state MASTER #节点状态
interface ens192 #节点keepalived接⼝IP地址
virtual_router_id 131 #虚拟路由器ID - - 主备节点参数需要相同
#尽可能⾃定义 1-255 ｜防⽌冲突
priority 150 #选举优先级 - Master节点优先级⾼
advert_int 1 # ⼼跳通告时间间隔 advertisement-internal - 1S |检测倍数3倍
authentication
auth_type PASS #开始认证
auth_pass Admin@123 #
virtual_ipaddress
192.168.8.113
track_script
keepalived_check_nginx_service_port
#测试流程
systemctl stop nginx && systemctl stop keepalived
systemctl start nginx && systemctl start keepalived
systemctl is-active nginx && systemctl is-active keepalived
active
active
[root@ha1 keepalived]# systemctl is-active nginx
active
[root@ha1 keepalived]# netstat -ntlp | grep nginx
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN
2238/nginx: master
[root@ha1 keepalived]# systemctl is-active keepalived.service
active
systemctl stop nginx
[root@ha1 keepalived]# systemctl is-active nginx
inactive
[root@ha1 keepalived]# systemctl status keepalived.service | tail -n 5
Aug 01 05:42:21 ha1 Keepalived_vrrp[2472]:
/etc/keepalived/check_nginx_service_port.sh exited with status 127
Aug 01 05:42:23 ha1 Keepalived_vrrp[2472]:
/etc/keepalived/check_nginx_service_port.sh exited with status 127
Aug 01 05:42:25 ha1 Keepalived_vrrp[2472]:
/etc/keepalived/check_nginx_service_port.sh exited with status 127
Aug 01 05:42:27 ha1 Keepalived_vrrp[2472]:
/etc/keepalived/check_nginx_service_port.sh exited with status 127
Aug 01 05:42:29 ha1 Keepalived_vrrp[2472]:
/etc/keepalived/check_nginx_service_port.sh exited with status 127
Aug 01 05:42:31 ha1 Keepalived_vrrp[2472]:
原因是selinux没关闭。
[root@ha1 ~]# sestatus -v | grep -i mode
Current mode: enforcing
Mode from config file: permissive
[root@ha1 ~]# setenforce 0
[root@ha1 ~]# sestatus -v | grep -i mode
Current mode: permissive
Mode from config file: permissive
#继续再次测试
[root@ha1 ~]# systemctl stop nginx
[root@ha1 ~]# systemctl is-active nginx
active
[root@ha1 ~]# systemctl status keepalived.service | tail -n 10
Aug 01 05:49:17 ha1 Keepalived_vrrp[9783]: Sending gratuitous ARP on ens192 for
192.168.8.113
Aug 01 05:49:22 ha1 Keepalived_vrrp[9783]: Sending gratuitous ARP on ens192 for
192.168.8.113
Aug 01 05:49:22 ha1 Keepalived_vrrp[9783]: VRRP_Instance(VI_1) Sending/queueing
gratuitous ARPs on ens192 for 192.168.8.113
Aug 01 05:49:22 ha1 Keepalived_vrrp[9783]: Sending gratuitous ARP on ens192 for
192.168.8.113
Aug 01 05:49:22 ha1 Keepalived_vrrp[9783]: Sending gratuitous ARP on ens192 for
192.168.8.113
Aug 01 05:49:22 ha1 Keepalived_vrrp[9783]: Sending gratuitous ARP on ens192 for
192.168.8.113
Aug 01 05:49:22 ha1 Keepalived_vrrp[9783]: Sending gratuitous ARP on ens192 for
192.168.8.113
Aug 01 05:49:48 ha1 Keepalived_vrrp[9783]:
VRRP_Script(keepalived_check_nginx_service_port) timed out
Aug 01 05:49:48 ha1 Keepalived_vrrp[9783]:
/etc/keepalived/check_nginx_service_port.sh exited due to signal 15
Aug 01 05:49:48 ha1 Keepalived_vrrp[9783]:
VRRP_Script(keepalived_check_nginx_service_port) succeeded
Keepalived-HA-Nginx-健康检测-备份节点
! Configuration File for keepalived
global_defs
notification_email
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1 #邮箱地址
smtp_connect_timeout 30
router_id keepalived_backup # 真实路由器ID - 主备节点参数需要不同
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
vrrp_script keepalived_check_nginx_service_port
script "/etc/keepalived/check_nginx_service_port.sh"
interval 2 # 执⾏脚本时间间隔
vrrp_instance VI_1
state BACKUP #节点状态
interface ens192 #节点keepalived接⼝IP地址
virtual_router_id 131 #虚拟路由器ID - - 主备节点参数需要相同
#尽可能⾃定义 1-255 ｜防⽌冲突
priority 90 #选举优先级 - Master节点优先级⾼
advert_int 1 # ⼼跳通告时间间隔 advertisement-internal - 1S |检测倍数3倍
authentication
auth_type PASS #开始认证
auth_pass Admin@123 #
virtual_ipaddress
192.168.8.113
track_script
keepalived_check_nginx_service_port
以上是关于Nginx反向代理-Keepalived做高可用的主要内容，如果未能解决你的问题，请参考以下文章
keepalived+nginx-upstream部署高可用反向代理