keepalived基于nginx实现双主模型
Posted
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了keepalived基于nginx实现双主模型相关的知识，希望对你有一定的参考价值。
准备环境
| Director 1 | Director 2 | Web Server nginx1 | Web Server Nginx |
| DIP:172.18.42.100 | DIP:172.18.42.22 | RIP:172.18.42.111 | RIP:172.18.42.222 |
VIP:172.18.42.119(MASTER) VIP:172.18.42.120(BACKUP) | VIP:172.18.42.119(BACKUP) VIP:172.18.42.120(MASTER) | VIP:172.18.42.119 VIP:172.18.42.120 | VIP:172.18.42.119 VIP:172.18.42.120 |
一、Director 1 配置
1、yum安装keepalived
[[email protected] ~]# yum install keepalived -y
2、编辑配置文件
[[email protected] ~]# vim /etc/keepalived/keepalived.conf global_defs { notification_email { [email protected] } notification_email_from www.mageedu.com smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id Slackware vrrp_mcast_group4 224.0.42.10 } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 110 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.18.42.119 dev eth0 label eth0:1 } } virtual_server 172.18.42.119 80 { delay_loop 6 lb_algo rr lb_kind DR protocol TCP real_server 172.18.42.111 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } real_server 172.18.42.222 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } } vrrp_instance VI_2 { state BACKUP interface eth0 virtual_router_id 111 priority 98 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.18.42.120 dev eth0 label eth0:2 } } virtual_server 172.18.42.120 80 { delay_loop 6 lb_algo rr lb_kind DR protocol TCP real_server 172.18.42.111 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } real_server 172.18.42.222 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } }
二、Director 2 配置
1、安装keepalived服务
[[email protected]~]# yum install keepalived -y
2、编辑配置文件
[[email protected] ~]# vim /etc/keepalived/keepalived.conf global_defs { notification_email { [email protected] } notification_email_from www.mageedu.com smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id Slackware vrrp_mcast_group4 224.0.42.10 } vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 110 priority 98 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.18.42.119 dev eth0 label eth0:1 } } virtual_server 172.18.42.119 80 { delay_loop 6 lb_algo rr lb_kind DR protocol TCP real_server 172.18.42.111 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } real_server 172.18.42.222 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } } vrrp_instance VI_2 { state MASTER interface eth0 virtual_router_id 111 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.18.42.120 dev eth0 label eth0:2 } } virtual_server 172.18.42.120 80 { delay_loop 6 lb_algo rr lb_kind DR protocol TCP real_server 172.18.42.111 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } real_server 172.18.42.222 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } }
三、Web Server Nginx1配置
1、修改Web Server Nginx1的内核参数
[[email protected] ~]# echo 1 &> /proc/sys/net/ipv4/conf/all/arp_ignore [[email protected] ~]# echo 1 &> /proc/sys/net/ipv4/conf/lo/arp_ignore [[email protected] ~]# echo 2 &> /proc/sys/net/ipv4/conf/lo/arp_announce [[email protected] ~]# echo 2 &> /proc/sys/net/ipv4/conf/all/arp_announce
2、添加VIP
[[email protected] ~]# ifconfig lo:1 172.18.42.119 netmask 255.255.255.255 broadcast 172.18.42.119 [[email protected] ~]# ifconfig lo:2 172.18.42.120 netmask 255.255.255.255 broadcast 172.18.42.120 [[email protected] ~]# route add -host 172.18.42.119 dev lo:1 [[email protected] ~]# route add -host 172.18.42.120 dev lo:2
3、安装nginx、并启动
[[email protected] ~]# yum install nginx -y [[email protected] ~]# nginx
四、Web Server Nginx2配置
1、修改Web Server Nginx2的内核参数
[[email protected] ~]# echo 1 &> /proc/sys/net/ipv4/conf/all/arp_ignore [[email protected] ~]# echo 1 &> /proc/sys/net/ipv4/conf/lo/arp_ignore [[email protected] ~]# echo 2 &> /proc/sys/net/ipv4/conf/lo/arp_announce [[email protected] ~]# echo 2 &> /proc/sys/net/ipv4/conf/all/arp_announce
2、添加VIP
[[email protected] ~]# ifconfig lo:1 172.18.42.119 netmask 255.255.255.255 broadcast 172.18.42.119 [[email protected] ~]# ifconfig lo:2 172.18.42.120 netmask 255.255.255.255 broadcast 172.18.42.120 [[email protected] ~]# route add -host 172.18.42.119 dev lo:1 [[email protected] ~]# route add -host 172.18.42.120 dev lo:2
3、安装nginx，并启动
[[email protected] ~]# yum install nginx -y [[email protected] ~]# nginx
五、开启两台Director，并测试
1、Director 1
[[email protected] ~]# service keepalived start [[email protected] ~]# ifconfig eth0 Link encap:Ethernet HWaddr 00:0C:29:90:84:C0 inet addr:172.18.42.100 Bcast:172.18.255.255 Mask:255.255.0.0 inet6 addr: fe80::20c:29ff:fe90:84c0/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:182021 errors:0 dropped:0 overruns:0 frame:0 TX packets:48384 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:28094276 (26.7 MiB) TX bytes:4476727 (4.2 MiB) eth0:1 Link encap:Ethernet HWaddr 00:0C:29:90:84:C0 inet addr:172.18.42.119 Bcast:0.0.0.0 Mask:255.255.255.255 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:663 errors:0 dropped:0 overruns:0 frame:0 TX packets:663 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:49597 (48.4 KiB) TX bytes:49597 (48.4 KiB) [[email protected] ~]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 172.18.42.119:80 rr -> 172.18.42.111:80 Route 1 0 0 -> 172.18.42.222:80 Route 1 0 1 TCP 172.18.42.120:80 rr -> 172.18.42.111:80 Route 1 0 0 -> 172.18.42.222:80 Route 1 0 0
## Director 1的VIP已经启动；同时ipvsadm的规则自动生成
2、Director 2
[[email protected] ~]# ifconfig eth0 Link encap:Ethernet HWaddr 00:0C:29:0F:A3:6E inet addr:172.18.42.22 Bcast:172.18.255.255 Mask:255.255.0.0 inet6 addr: fe80::20c:29ff:fe0f:a36e/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:134293 errors:0 dropped:0 overruns:0 frame:0 TX packets:69576 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:16446378 (15.6 MiB) TX bytes:5481841 (5.2 MiB) eth0:2 Link encap:Ethernet HWaddr 00:0C:29:0F:A3:6E inet addr:172.18.42.120 Bcast:0.0.0.0 Mask:255.255.255.255 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:3431 errors:0 dropped:0 overruns:0 frame:0 TX packets:3431 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:291422 (284.5 KiB) TX bytes:291422 (284.5 KiB) [[email protected] ~]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 172.18.42.119:80 rr -> 172.18.42.111:80 Route 1 0 0 -> 172.18.42.222:80 Route 1 0 0 TCP 172.18.42.120:80 rr -> 172.18.42.111:80 Route 1 0 0 -> 172.18.42.222:80 Route 1 0 0
试着访问Web服务
[[email protected] ~]# curl http://172.18.42.119 <h1> 172.18.42.222 Web Server 2 </h1> [[email protected] ~]# curl http://172.18.42.119 <h1> 172.18.42.111 Web Server 1 </h1> [[email protected] ~]# curl http://172.18.42.120 <h1> 172.18.42.222 Web Server 2 </h1> [[email protected] ~]# curl http://172.18.42.120 <h1> 172.18.42.111 Web Server 1
##实现了负载均衡、双主模型
六、把Director 1关闭掉；会发生什么样的情况？
Director 1
[[email protected] ~]# service keepalived stop
Director 2
[[email protected] ~]# ifconfig eth0 Link encap:Ethernet HWaddr 00:0C:29:0F:A3:6E inet addr:172.18.42.22 Bcast:172.18.255.255 Mask:255.255.0.0 inet6 addr: fe80::20c:29ff:fe0f:a36e/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:142076 errors:0 dropped:0 overruns:0 frame:0 TX packets:73562 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:17381764 (16.5 MiB) TX bytes:5790752 (5.5 MiB) eth0:1 Link encap:Ethernet HWaddr 00:0C:29:0F:A3:6E inet addr:172.18.42.119 Bcast:0.0.0.0 Mask:255.255.255.255 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 eth0:2 Link encap:Ethernet HWaddr 00:0C:29:0F:A3:6E inet addr:172.18.42.120 Bcast:0.0.0.0 Mask:255.255.255.255 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:3435 errors:0 dropped:0 overruns:0 frame:0 TX packets:3435 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:291770 (284.9 KiB) TX bytes:291770 (284.9 KiB) [[email protected] ~]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 172.18.42.119:80 rr -> 172.18.42.111:80 Route 1 0 0 -> 172.18.42.222:80 Route 1 0 0 TCP 172.18.42.120:80 rr -> 172.18.42.111:80 Route 1 0 1 -> 172.18.42.222:80 Route 1 0 1
##当把Director 1关闭掉之后，此时Director 2会把Director的VIP夺过来
我们再来访问Web服务
[[email protected] ~]# curl http://172.18.42.120 <h1> 172.18.42.222 Web Server 2 </h1> [[email protected] ~]# curl http://172.18.42.120 <h1> 172.18.42.111 Web Server 1 </h1> [[email protected] ~]# curl http://172.18.42.119 <h1> 172.18.42.222 Web Server 2 </h1> [[email protected] ~]# curl http://172.18.42.119 <h1> 172.18.42.111 Web Server 1 </h1>
##服务能够正常提供
七、如果Web2服务挂掉了，会发生什么样的情况？
Web Server Nginx2
[[email protected] ~]# nginx -s stop
Director
[[email protected] ~]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 172.18.42.119:80 rr -> 172.18.42.222:80 Route 1 0 1 TCP 172.18.42.120:80 rr -> 172.18.42.222:80 Route 1 0 1
##服务器自动剔除Web1的IP
再次尝试访问Web服务
[[email protected] ~]# curl http://172.18.42.119 <h1> 172.18.42.222 Web Server 2 </h1> [[email protected] ~]# curl http://172.18.42.119 <h1> 172.18.42.222 Web Server 2 </h1> [[email protected] ~]# curl http://172.18.42.120 <h1> 172.18.42.222 Web Server 2 </h1> [[email protected] ~]# curl http://172.18.42.120 <h1> 172.18.42.222 Web Server 2 </h1>
##响应请求的只有Web Server Nginx2了
如果开启Web Server Nginx1；服务器会自动添加规则吗？
[[email protected] ~]# nginx [[email protected] ~]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 172.18.42.119:80 rr -> 172.18.42.111:80 Route 1 0 0 -> 172.18.42.222:80 Route 1 0 0 TCP 172.18.42.120:80 rr -> 172.18.42.111:80 Route 1 0 0 -> 172.18.42.222:80 Route 1 0 0
##Web Server Nginx1的规则自动添加
问题总结：
1、对于Director中的keepalived配置文件；两个vrrp示例的“vrrp_router_id”必须不能一样
2、每台Director的时间需要同步；可基于ntpdate实现
本文出自 “wtc” 博客，请务必保留此出处http://wangtianci.blog.51cto.com/11265133/1774047
以上是关于keepalived基于nginx实现双主模型的主要内容，如果未能解决你的问题，请参考以下文章
单主模型的IPVS高可用和nginx+keepalived的双主高可用模型