LVS集群的组成
利用LVS架设的服务器集群系统由3个部分组成:最前端的是负载均衡层(HA Load Balance),中间是服务器群组层(Server Array),底端是数据共享存储层(Shared Storage)。
LVS集群的特点
LVS提供负载均衡,keepalived提供健康检查,故障转移,提供高可用性!
LVS的IP负载均衡技术是通过VIP模块来实现的。IPVS是LVS集群系统的核心软件,它主要作用是:安装在Director Server上,同时在Director Server上虚拟出一个IP地址,用户必须通过这个虚拟IP地址访问服务器。这个虚拟IP一般称为LVS的VIP,即Virtual IP。访问的请求首先经过VIP到达负载调度器,然后由负载调度器从Real Server列表中选取一个服务节点响应用户的请求。
支持TCP协议的应用有:HTTP、HTTPS、FTP、SMTP、POP3、IMAP4、PROXY、LDAP和SSMTP等;支持UDP协议应用有:DNS、NTP、ICP、视频和音频流播放协议等。
IPVS实现负载均衡的方法有三种:
VS/NAT:即Virtual Server via Network Address Translation,也就是网络地址翻译技术实现虚拟服务器。
VS/TUN:即Virtual Server via IP Tunneling,也就是通过IP隧道技术实现虚拟服务器。
VS/DR:即Virtual Server via Direct Routing,也就是且直接路由技术实现虚拟服务器。
DR是这三种负载调度方式中性能最好的,但是要求Director Server与Real Server必须连在同一物理网段上。
LVS DR原理:
用户请求LVS到达director,director将请求的报文目标MAC地址改为后端realserver MAC地址,目标IP为VIP(不变),源IP为用户IP,然后director将报文发送到realserver,realserver检测到目标为自己本地VIP,然后将回应直接发给用户(转发效率最高)
LVS NAT原理:
用户请求LVS到达directory,directory将请求的报文目标IP地址改为后端realserver IP地址,同时将报文的目标端口也改为后端选定的realserver相应端口,最后将报文发送到realserver,realserver将数据回应给directory,directory再将数据发送给用户
负载调度算法共有8种,下面列举最常用的4种调度算法:
轮叫调度(Round Robin)、加权轮叫调度(Weighted Round Robin)、最少连接高度(Least Connection)、加权最少连接调度(Weighted Least Coonection)。
IP规划
lvs-master:192.168.200.101
lvs-backup:192.168.200.102
vip:192.168.200.100
realserver1:192.168.200.103
realserver2:192.168.200.104
netmask:255.255.255.0
gateway:192.168.200.2
自动调整系统时间
yum install -y ntp
ntpdate time1.aliyun.com
echo "00 01 * * * /usr/sbin/ntpdate time1.aliyun.com" >>/etc/crontab
配置ipvsadm和keepalived
在LVS-Master上:
-
安装ipvsadm和keepalived
yum install -y gcc gcc-c++ wget libnl-devel popt-devel openssl-devel popt-static libnfnetlink libnfnetlink-devel kernel-devel
reboot
cd /usr/local/src
wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.26.tar.gz
wget http://www.keepalived.org/software/keepalived-1.3.4.tar.gz
mkdir /usr/src/linux
ln -sv /usr/src/kernels/3.10.0-514.el7.x86_64/ /usr/src/linux/
tar -zxvf ipvsadm-1.26.tar.gz
cd ipvsadm-1.26
make && make install
cd ..
tar -xzvf keepalived-1.3.4.tar.gz
cd keepalived-1.3.4
./configure
make && make install
编译的时候出现下面的提示,说明keepalived和内核结合了,否则需要加上这个参数./configure --with-kernel-dir=/kernel/path
2. 配置keepalived
cp /usr/local/src/keepalived-1.3.4/keepalived/etc/init.d/keepalived /etc/init.d/
cp /usr/local/src/keepalived-1.3.4/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
mkdir /etc/keepalived/
cp /usr/local/src/keepalived-1.3.4/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
cp /usr/local/src/keepalived-1.3.4/bin/keepalived /usr/sbin/
vim /etc/keepalived/keepalived.conf
!Configuration File for keepalived
global_defs {
notification_email {
}
notification_email_from [email protected]
smtp_server smtp.domain.com
#smtp_connect_timeout 30
router_id LVS_MASTER #设置lvs的id,在一个网络内必须唯一
}
#VIP1
vrrp_instance VI_1 {
state MASTER #备份服务器上将MASTER改为BACKUP
interface ens32
lvs_sync_daemon_inteface ens32
virtual_router_id 51 #虚拟路由标示
priority 100 #备份服务上将100改为90
advert_int 5 #主从服务器间检查间隔秒数
authentication {
auth_type PASS #验证模式有PASS和HA两种
auth_pass 1111
}
virtual_ipaddress {
192.168.9.200 #如果有多个VIP,继续换行填写。
}
}
virtual_server 192.168.9.200 80 {
delay_loop 6 #每隔6秒查询realserver状态
lb_algo wlc #lvs 算法
lb_kind DR #Direct Route
persistence_timeout 60 #同一IP的连接60秒内被分配到同一台realserver
protocol TCP #TCP协议检查realserver状态
real_server 192.168.200.101 80 {
weight 100 #权重
TCP_CHECK {
connect_timeout 10 #10秒无响应超时
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 192.168.200.102 80 {
weight 100
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
systemctl restart keepalived
systemctl enable keepalived
3. 在LVS-Backup上:
Backup机的配置和Master机大致相同,仅有三点不同:
router_id LVS_BACKUP
state BACKUP
priority 90
4. 配置RealServer
RealServer1
yum install -y httpd httpd-devel
vim /etc/httpd/conf/httpd.conf
95 ServerName 192.168.200.103:80
echo "web1 2 3 4 5" > /var/www/html/index.html
RealServer2
yum install -y httpd httpd-devel
vim /etc/httpd/conf/httpd.conf
95 ServerName 192.168.200.104:80
echo "web2 1 2 3 4 5" > /var/www/html/index.html
RealServer2和RealServer1脚本部分配置相同
vim /root/lvs-real.sh
#!/bin/bash
SNS_VIP=192.168.200.100
. /etc/rc.d/init.d/functions
case "$1" in
start)
/sbin/ifconfig lo:0 $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP
/sbin/route add -host $SNS_VIP dev lo:0
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
sysctl -p >/dev/null 2>&1
echo "RealServer Start OK"
;;
stop)
/sbin/ifconfig lo:0 down
/sbin/route del $SNS_VIP >/dev/null 2>&1
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "RealServer Stoped"
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac
exit 0
chmod u+x /root/lvs-real.sh
/root/lvs-real.sh start
chmod u+rwx /etc/rc.d/rc.local
echo “/root/lvs-real.sh start”>>/etc/rc.d/rc.local
ifconfig
5. 测试LVS+keepalived
LVS_MASTER
LVS_BACKUP
客户端测试
在LVS_master上查看客户端连接情况
关闭LVS_master,短暂的掉包后,LVS_backup马上接替工作,LVS_backup接替LVS_master绑定VIP。
LVS_backup负责转发。
LVS_master重启完成后,就会自动接回控制权,继续负责转发。
LVS_master
LVS_backup
测试关闭其中一台realserver。
当realserver故障或者无法提供服务时,负载均衡器通过健康检查自动把失效的机器从转发队列删除掉,实现故障隔离,保证用户的访问不受影响。
重启被关闭的realserver1
当realserver1故障恢复后,负载均衡器通过健康检查自动把恢复后的机器添加到转发队列中。