Nginx反向代理-Keepalived做高可用

Posted

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了Nginx反向代理-Keepalived做高可用相关的知识,希望对你有一定的参考价值。

Keepalived-HA-主节点配置管理

1.软件安装

[root@ha1 nginx]# yum install -y keepalived
[root@ha1 nginx]# cd /etc/keepalived/
[root@ha1 keepalived]# mv keepalived.conf keepalived.conf.bak


2.主备配置-集群功能

[root@ha1 keepalived]# vim keepalived.conf
! Configuration File for keepalived

global_defs
notification_email
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc

notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1 #邮箱地址
smtp_connect_timeout 30
router_id keepalived_master # 真实路由器ID - 主备节点参数需要不同
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0


vrrp_instance VI_1
state MASTER #节点状态
interface ens33 #节点keepalived接⼝IP地址
virtual_router_id 131 #虚拟路由器ID - - 主备节点参数需要相同
#尽可能⾃定义 1-255 |防⽌冲突
priority 150 #选举优先级 - Master节点优先级⾼
advert_int 1 # ⼼跳通告时间间隔 advertisement-internal - 1S |检测倍数3倍
authentication
auth_type PASS #开始认证
auth_pass Admin@123 #

virtual_ipaddress
192.168.40.113


3.启动与自启动
[root@ha1 keepalived]# systemctl start keepalived.service && systemctl enable keepalived.service
[root@ha1 keepalived]# systemctl status keepalived.service
4.查看网址信息

[root@ha1 keepalived]# ip add
2: ens33: mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:15:8f:0c brd ff:ff:ff:ff:ff:ff
inet 192.168.40.111/24 brd 192.168.40.255 scope global ens33 #本机⽹卡真实地址
valid_lft forever preferred_lft forever
inet 192.168.40.113/32 scope global ens33 # 集群虚拟IP地址 (主)


Keepalived-HA-备节点配置管理

1.软件安装

[root@ha2 nginx]# yum install -y keepalived
[root@ha2 nginx]# cd /etc/keepalived/
[root@ha2 keepalived]# mv keepalived.conf keepalived.conf.bak


2.文件配置

[root@ha2 keepalived]# vim keepalived.conf
! Configuration File for keepalived

global_defs
notification_email
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc

notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1 #邮箱地址
smtp_connect_timeout 30
router_id keepalived_backup # 真实路由器ID - 主备节点参数需要不同
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0


vrrp_instance VI_1
state BACKUP #节点状态
interface ens33 #节点keepalived接⼝IP地址
virtual_router_id 131 #虚拟路由器ID - 主备节点参数需要相同
#尽可能⾃定义 1-255 |防⽌冲突
priority 90 #选举优先级 - Master节点优先级⾼
advert_int 1 # ⼼跳通告时间间隔 advertisement-internal - 1S |检测倍数3倍
authentication
auth_type PASS #开始认证
auth_pass Admin@123 #

virtual_ipaddress
192.168.40.113


3、启动与自启动

[root@ha2 keepalived]# systemctl start keepalived.service && systemctl enable keepalived.service
[root@ha2 keepalived]# systemctl status keepalived.service


4、查看网址信息

[root@ha2 keepalived]# ip add 2: ens33: mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:30:ed:19 brd ff:ff:ff:ff:ff:ff inet 192.168.40.112/24 brd 192.168.40.255 scope global ens33


Keepalived-HA-主备倒换测试概述

主备倒换测试

1.模拟主节点故障 - 观察,⽤户访问流量是否能正常切换到备份节点。 ----- Keepalived服务停⽌

2.恢复主节点 - 观察,⽤户访问流量是否能正常回切到主节点。 ----- Keepalived服务正常打开


Keepalived-HA-主备倒换测试操作

[root@ha1 keepalived]# systemctl stop keepalived.service
root@ha2 keepalived]# systemctl status keepalived.service
● keepalived.service - LVS and VRRP High Availability Monitor
Loaded: loaded (/usr/lib/systemd/system/keepalived.service; enabled; vendor preset: disabled)
Active: active (running) since Fri 2022-12-23 10:27:48 CST; 4h 20min ago
Main PID: 14663 (keepalived)
CGroup: /system.slice/keepalived.service
├─14663 /usr/sbin/keepalived -D
├─14664 /usr/sbin/keepalived -D
└─14665 /usr/sbin/keepalived -D

Dec 23 14:48:17 ha2 Keepalived_vrrp[14665]: VRRP_Instance(VI_1) Transition to MASTER STATE
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: VRRP_Instance(VI_1) Entering MASTER STATE #就是这个位置
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: VRRP_Instance(VI_1) setting protocol iptable drop rule
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: VRRP_Instance(VI_1) setting protocol VIPs.
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: Sending gratuitous ARP on ens33 for 192.168.40.113
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ens33 for 192.168.40.113
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: Sending gratuitous ARP on ens33 for 192.168.40.113
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: Sending gratuitous ARP on ens33 for 192.168.40.113
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: Sending gratuitous ARP on ens33 for 192.168.40.113
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: Sending gratuitous ARP on ens33 for 192.168.40.113


Keepalived-HA-Nginx-健康检测

[root@ha1 keepalived]# systemctl stop nginx
[root@ha1 keepalived]#
[root@ha1 keepalived]# ip add show ens192
2: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen
1000
link/ether 00:50:56:94:4a:c5 brd ff:ff:ff:ff:ff:ff
inet 192.168.40.111/24 brd 192.168.8.255 scope global ens192
valid_lft forever preferred_lft forever
inet 192.168.8.113/32 scope global ens192 #集群IP地址
valid_lft forever preferred_lft forever

# 当Nginx⽆法提供服务时,当前节点⾃动关闭keepalived

# 主节点Nginx⽆法提供服务时 ,keepalived⾃动关闭 - 不再发送⼼跳消息 - 备份节点如果nginx可

⽤切换keepalived主节点。


1、抓取个数

[root@ha1 keepalived]# netstat -ntlp | grep "nginx: master" | wc -l
1 #nginx端⼝监听成功
[root@ha2 keepalived]# netstat -ntlp | grep "nginx: master" | wc -l
0 #nginx端⼝监听失败

1.检测监听端⼝

2.监听没有开启 - 尝试拉活

3.如果⽆法拉活 - 关闭keepalived


2、自启脚本

cd /etc/keepalived
vim check_nginx_service_port.sh
#!/bin/bash
if [ "$(netstat -ntlp | grep "nginx: master" | wc -l)" == "0" ]
then
systemctl restart nginx
sleep 2
if [ "$(netstat -ntlp | grep "nginx: master" | wc -l)" == "0" ]
then
systemctl stop keepalive
fi
fi
[root@ha1 keepalived]# chmod +x check_nginx_service_port.sh


! Configuration File for keepalived

global_defs
notification_email
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc

notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1 #邮箱地址
smtp_connect_timeout 30
router_id keepalived_master # 真实路由器ID - 主备节点参数需要不同
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0


vrrp_script keepalived_check_nginx_service_port
script "/etc/keepalived/check_nginx_service_port.sh"
interval 2 # 执⾏脚本时间间隔


vrrp_instance VI_1
state MASTER #节点状态
interface ens192 #节点keepalived接⼝IP地址
virtual_router_id 131 #虚拟路由器ID - - 主备节点参数需要相同
#尽可能⾃定义 1-255 |防⽌冲突
priority 150 #选举优先级 - Master节点优先级⾼
advert_int 1 # ⼼跳通告时间间隔 advertisement-internal - 1S |检测倍数3倍
authentication
auth_type PASS #开始认证
auth_pass Admin@123 #


virtual_ipaddress
192.168.8.113


track_script
keepalived_check_nginx_service_port



#测试流程

systemctl stop nginx && systemctl stop keepalived
systemctl start nginx && systemctl start keepalived
systemctl is-active nginx && systemctl is-active keepalived
active
active

Nginx反向代理-Keepalived做高可用_nginx


[root@ha1 keepalived]# systemctl is-active nginx
active
[root@ha1 keepalived]# netstat -ntlp | grep nginx
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN
2238/nginx: master
[root@ha1 keepalived]# systemctl is-active keepalived.service
active


Nginx反向代理-Keepalived做高可用_nginx_02


systemctl stop nginx
[root@ha1 keepalived]# systemctl is-active nginx
inactive
[root@ha1 keepalived]# systemctl status keepalived.service | tail -n 5
Aug 01 05:42:21 ha1 Keepalived_vrrp[2472]:
/etc/keepalived/check_nginx_service_port.sh exited with status 127
Aug 01 05:42:23 ha1 Keepalived_vrrp[2472]:
/etc/keepalived/check_nginx_service_port.sh exited with status 127
Aug 01 05:42:25 ha1 Keepalived_vrrp[2472]:
/etc/keepalived/check_nginx_service_port.sh exited with status 127
Aug 01 05:42:27 ha1 Keepalived_vrrp[2472]:
/etc/keepalived/check_nginx_service_port.sh exited with status 127
Aug 01 05:42:29 ha1 Keepalived_vrrp[2472]:
/etc/keepalived/check_nginx_service_port.sh exited with status 127
Aug 01 05:42:31 ha1 Keepalived_vrrp[2472]:

原因是selinux没关闭。

[root@ha1 ~]# sestatus -v | grep -i mode
Current mode: enforcing
Mode from config file: permissive

[root@ha1 ~]# setenforce 0
[root@ha1 ~]# sestatus -v | grep -i mode
Current mode: permissive
Mode from config file: permissive


#继续再次测试
[root@ha1 ~]# systemctl stop nginx
[root@ha1 ~]# systemctl is-active nginx
active
[root@ha1 ~]# systemctl status keepalived.service | tail -n 10
Aug 01 05:49:17 ha1 Keepalived_vrrp[9783]: Sending gratuitous ARP on ens192 for
192.168.8.113
Aug 01 05:49:22 ha1 Keepalived_vrrp[9783]: Sending gratuitous ARP on ens192 for
192.168.8.113
Aug 01 05:49:22 ha1 Keepalived_vrrp[9783]: VRRP_Instance(VI_1) Sending/queueing
gratuitous ARPs on ens192 for 192.168.8.113
Aug 01 05:49:22 ha1 Keepalived_vrrp[9783]: Sending gratuitous ARP on ens192 for
192.168.8.113
Aug 01 05:49:22 ha1 Keepalived_vrrp[9783]: Sending gratuitous ARP on ens192 for
192.168.8.113
Aug 01 05:49:22 ha1 Keepalived_vrrp[9783]: Sending gratuitous ARP on ens192 for
192.168.8.113
Aug 01 05:49:22 ha1 Keepalived_vrrp[9783]: Sending gratuitous ARP on ens192 for
192.168.8.113
Aug 01 05:49:48 ha1 Keepalived_vrrp[9783]:
VRRP_Script(keepalived_check_nginx_service_port) timed out
Aug 01 05:49:48 ha1 Keepalived_vrrp[9783]:
/etc/keepalived/check_nginx_service_port.sh exited due to signal 15
Aug 01 05:49:48 ha1 Keepalived_vrrp[9783]:
VRRP_Script(keepalived_check_nginx_service_port) succeeded


Keepalived-HA-Nginx-健康检测-备份节点


! Configuration File for keepalived
global_defs
notification_email
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc

notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1 #邮箱地址
smtp_connect_timeout 30
router_id keepalived_backup # 真实路由器ID - 主备节点参数需要不同
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0

vrrp_script keepalived_check_nginx_service_port
script "/etc/keepalived/check_nginx_service_port.sh"
interval 2 # 执⾏脚本时间间隔

vrrp_instance VI_1
state BACKUP #节点状态
interface ens192 #节点keepalived接⼝IP地址
virtual_router_id 131 #虚拟路由器ID - - 主备节点参数需要相同
#尽可能⾃定义 1-255 |防⽌冲突
priority 90 #选举优先级 - Master节点优先级⾼
advert_int 1 # ⼼跳通告时间间隔 advertisement-internal - 1S |检测倍数3倍
authentication
auth_type PASS #开始认证
auth_pass Admin@123 #

virtual_ipaddress
192.168.8.113

track_script
keepalived_check_nginx_service_port


以上是关于Nginx反向代理-Keepalived做高可用的主要内容,如果未能解决你的问题,请参考以下文章

Nginx反向代理负载均衡, keepalived高可用

keepalived实现高可用nginx反向代理

keepalived+nginx-upstream部署高可用反向代理

Nginx反向代理,负载均衡,redis session共享,keepalived高可用

Keepalived实现高可用Nginx反向代理

centos7 Nginx+keepalived高可用+反向代理