CentOS 7 部署LVS集群(DR模式NAT模式,LVS+keepalived)
Posted
tags:
篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了CentOS 7 部署LVS集群(DR模式NAT模式,LVS+keepalived)相关的知识,希望对你有一定的参考价值。
1.简介1.1LVS
linux虚拟服务器,是一个虚拟的服务器集群系统,可以在Unix和linux平台下实现负载均衡集群的功能。
1.2LVS与nginx的对比
1)工作在网络模型的7层,可以针对http应用做一些分流的策略,nginx单凭这点可利用的场合远多于LVS。
2)最新版的nginx也支持4层TCP负载,这曾是LVS的优势。
3)nginx对网络稳定性的依赖小,相反LVS对网络稳定性依赖大。
4)nginx安装和配置比较简单,测试起来比较方便,基本上能把错误用日志打印出来。LVS的配置、测试就要花比较长的时间,对网络依赖较大。
1.3搭建LVS的原因
1)日1000-2000W PV或并发请求1万以下都可考虑nginx
2)大型门户网站、电商网站需要用到LVS。
1.4LVS集群的工作模式
1)DR模式:通过改写请求报文的目标MAC地址,将请求发送给真实服务器,而真实服务器将相应后的处理结果直接返还给客户端用户。极大地提高集群的伸缩性,但LB与RS必须在同一局域网环境。
2)NAT模式:
通过网络地址转换,LB重写请求报文的目标地址,根据预设的调度算法,将请求分派给后端的真实服务器,真实服务器的响应报文处理之后,返回时必须通过LB,经过LB时报文的源地址被重写,再返回给客户。
3)隧道模式:LB把请求的报文通过IP隧道转发至真实服务器,而真实服务器将响应处理后直接返回给客户端用户。
4)FULLNAT模式:数据包进入时,除了做DNAT,还做SNAT,从而实现LVS-RealServer间可以跨vlan通讯,RealServer只需要连接到内网。
2.环境准备
1)关闭防火墙和Selinux
2)主机名及IP地址关系如下:
lb01 10.0.0.5
lb042 10.0.0.6
web03 10.0.0.17
web04 10.0.0.18
3)web03和web04安装Tomcat软件,并满足下面条件:
curl http://10.0.0.17 页面底部得到结果为web03
curl http://10.0.0.18 页面底部得到结果为web04
3.ipvsadm管理LVS负载集群(DR模式)
3.1安装ipvsadm管理工具(只在lb01操作)
[[email protected] ~]# yum install -y ipvsadm
说明:默认没有加载模块,需要安装管理工具才能激活
3.2查看LVS状态,并激活LVS内核模块
[[email protected] ~]# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
[[email protected] ~]# lsmod | grep ip_vs
ip_vs_rr 12600 0
ip_vs 140944 2 ip_vs_rr
nf_conntrack 105745 1 ip_vs
libcrc32c 12644 2 xfs,ip_vs
3.3在eth0网卡绑定VIP地址(只在lb01操作)
[[email protected] ~]# ip addr add 10.0.0.13/24 dev eth0
[[email protected] ~]# ip a s eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:a6:ba:9e brd ff:ff:ff:ff:ff:ff
inet 10.0.0.5/24 brd 10.0.0.255 scope global eth0
valid_lft forever preferred_lft forever
inet 10.0.0.13/24 scope global secondary eth0
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fea6:ba9e/64 scope link
valid_lft forever preferred_lft forever
3.4清除所有LVS规则(只在lb01操作)
[[email protected] ~]# ipvsadm -C
3.5设置tcp、tcpfin、udp连接超时时间
[[email protected] ~]# ipvsadm --set 30 5 60
3.6添加虚拟服务(只在lb01操作)
[[email protected] ~]# ipvsadm -A -t 10.0.0.13:80 -s wrr -p 20
3.7将虚拟服务关联到真实服务上(只在lb01操作)
[[email protected] ~]# ipvsadm -a -t 10.0.0.13:80 -r 10.0.0.17:80 -g -w 1
[[email protected] ~]# ipvsadm -a -t 10.0.0.13:80 -r 10.0.0.18:80 -g -w 1
[[email protected] ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.13:80 wrr persistent 20
-> 10.0.0.17:80 Route 1 0 0
-> 10.0.0.18:80 Route 1 0 0
3.8在lo网卡绑定VIP地址(web03、web04同时操作)
ip addr add 10.0.0.13/24 dev lo
3.9修改内核参数抑制ARP响应(web03、web04同时操作)
cat /etc/sysctl.conf
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
3.10浏览器访问http://10.0.0.13
4.LVS+keepalived(DR模式)
4.1安装keepalived(lb01、lb02同时操作)
yum install -y keepalived
4.2修改lb01的keepalived配置文件
[[email protected] ~]# cat /etc/keepalived/keepalived.conf
global_defs {
router_id LVS_01
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.13/24
}
}
virtual_server 10.0.0.13 80 {
delay_loop 6
lb_algo wrr
lb_kind DR
nat_mask 255.255.255.0
persistence_timeout 50
protocol TCP
real_server 10.0.0.17 80 {
weight 1
TCP_CHECK {
connect_timeout 8
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 10.0.0.18 80 {
weight 1
TCP_CHECK {
connect_timeout 8
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
4.3修改lb02的keepalived配置文件
[[email protected] ~]# cat /etc/keepalived/keepalived.conf
global_defs {
router_id LVS_02
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.13/24
}
}
virtual_server 10.0.0.13 80 {
delay_loop 6
lb_algo wrr
lb_kind DR
nat_mask 255.255.255.0
persistence_timeout 50
protocol TCP
real_server 10.0.0.17 80 {
weight 1
TCP_CHECK {
connect_timeout 8
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 10.0.0.18 80 {
weight 1
TCP_CHECK {
connect_timeout 8
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
4.4启动keepalived
systemctl start keepalived.service
4.5测试keepalived高可用,故障转移
4.6在lo网卡绑定VIP地址(web03、web04同时操作)
ip addr add 10.0.0.13/24 dev lo
4.7修改内核参数抑制ARP响应(web03、web04同时操作)
cat /etc/sysctl.conf
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
4.8浏览器访问http://10.0.0.13(测试keepalived健康检查)
5.ipvsadm管理LVS负载集群(NAT模式)
5.1安装ipvsadm管理工具(只在lb01操作)
[[email protected] ~]# yum install -y ipvsadm
说明:默认没有加载模块,需要安装管理工具才能激活
5.2查看LVS状态,并激活LVS内核模块
[[email protected] ~]# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
[[email protected] ~]# lsmod | grep ip_vs
ip_vs_rr 12600 0
ip_vs 140944 2 ip_vs_rr
nf_conntrack 105745 1 ip_vs
libcrc32c 12644 2 xfs,ip_vs
5.3在eth0网卡绑定VIP地址(只在lb01操作)
[[email protected] ~]# ip addr add 10.0.0.13/24 dev eth0
[[email protected] ~]# ip a s eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:a6:ba:9e brd ff:ff:ff:ff:ff:ff
inet 10.0.0.5/24 brd 10.0.0.255 scope global eth0
valid_lft forever preferred_lft forever
inet 10.0.0.13/24 scope global secondary eth0
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fea6:ba9e/64 scope link
valid_lft forever preferred_lft forever
5.4清除所有LVS规则(只在lb01操作)
[[email protected] ~]# ipvsadm -C
5.5设置tcp、tcpfin、udp连接超时时间(只在lb01操作)
[[email protected] ~]# ipvsadm --set 30 5 60
5.6添加虚拟服务(只在lb01操作)
[[email protected] ~]# ipvsadm -A -t 10.0.0.13:80 -s rr -p 20
5.7将虚拟服务关联到真实服务上(只在lb01操作)
ipvsadm -a -t 10.0.0.13:80 -r 172.16.1.203:80 -m
ipvsadm -a -t 10.0.0.13:80 -r 172.16.1.204:80 -m
5.9开启路由转发功能(只在lb01操作)
[[email protected] ~]# echo 'net.ipv4.ip_forward = 1' >> /etc/sysctl.conf
[[email protected] ~]# sysctl -p
5.10清空iptables规则(只在lb01操作)
[[email protected] ~]# iptables -F
[[email protected] ~]# iptables -F -t nat
5.11添加nat转换规则(只在lb01操作)
[[email protected] ~]# iptables -t nat -A POSTROUTING -s 172.16.1.0/24 -j MASQUERADE
5.12RS服务器内网网卡添加虚拟网关(web03、web04同时操作)
echo "GATEWAY=172.16.1.13" >> /etc/sysconfig/network-scripts/ifcfg-eth1
5.13浏览器访问http://10.0.0.15
6.LVS+keepalived(NAT模式)
6.1安装keepalived(lb01、lb02同时操作)
yum install -y keepalived
6.2修改lb01的keepalived配置文件
[[email protected] ~]# cat /etc/keepalived/keepalived.conf
global_defs {
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.13/24 dev eth0 label eth0:1
}
}
vrrp_instance VI_2 {
state MASTER
interface eth1
virtual_router_id 52
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass 1112
}
virtual_ipaddress {
172.16.1.13/24 dev eth1 label eth1:1
}
}
virtual_server 10.0.0.13 80 {
delay_loop 6
lb_algo rr
lb_kind NAT
nat_mask 255.255.255.0
persistence_timeout 50
protocol TCP
real_server 172.16.1.17 80 {
weight 1
TCP_CHECK {
connect_timeout 8
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 172.16.1.18 80 {
weight 1
TCP_CHECK {
connect_timeout 8
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
6.3修改lb02的keepalived配置文件
[[email protected] ~]# cat /etc/keepalived/keepalived.conf
global_defs {
router_id LVS_DEVEL1
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.13/24 dev eth0 label eth0:1
}
}
vrrp_instance VI_2 {
state BACKUP
interface eth1
virtual_router_id 52
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1112
}
virtual_ipaddress {
172.16.1.13/24 dev eth1 label eth1:1
}
}
virtual_server 10.0.0.13 80 {
delay_loop 6
lb_algo rr
lb_kind NAT
nat_mask 255.255.255.0
persistence_timeout 50
protocol TCP
real_server 172.16.1.17 80 {
weight 1
TCP_CHECK {
connect_timeout 8
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 172.16.1.18 80 {
weight 1
TCP_CHECK {
connect_timeout 8
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
6.4开启路由转发功能(lb01、lb02同时操作)
echo 'net.ipv4.ip_forward = 1' >> /etc/sysctl.conf
sysctl -p
6.5清空iptables规则(lb01、lb02同时操作)
iptables -F
iptables -F -t nat
6.6添加nat转换规则(lb01、lb02同时操作)
iptables -t nat -A POSTROUTING -s 172.16.1.0/24 -j MASQUERADE
6.7防止keepalived脑裂(lb01、lb02同时操作)
iptables -A INPUT -i eth0 -p vrrp -j ACCEPT
6.8启动keepalived
systemctl start keepalived.service
6.8RS服务器内网网卡添加虚拟网关(web03、web04同时操作)
echo "GATEWAY=172.16.1.13" >> /etc/sysconfig/network-scripts/ifcfg-eth1
6.9浏览器访问http://10.0.0.13
以上是关于CentOS 7 部署LVS集群(DR模式NAT模式,LVS+keepalived)的主要内容,如果未能解决你的问题,请参考以下文章
Cluster基础:ipvsadm命令用法部署LVS-NAT集群部署LVS-DR集群
CentOS7---基于 CentOS 7 构建 LVS-DR 群集