lvs + keepalived + nginx 实现高可用

Posted 2021-01-23

1.目的

 lvs是四层的负载均衡，keepalived为lvs提供高可用服务，同时检查后端nginx的健康状态，nginx主要用来做七层的负载均衡

2.拓扑图

服务器 IP地址    说明
Director主节点 192.168.3.105  lvs keepalived
Director备节点 192.168.3.104   lvs keepalived
Real server1：   192.168.3.106:89    nginx
Real server1：   192.168.3.107:89    nginx
vip     192.168.3.114   vip只需要还没有分配出去的ip，不需要分配磁盘、cpu

3.软件

keepalived-1.4.2.tar.gz
ipvsadm-1.27-7.el7.x86_64.rpm
nginx-1.12.2.tar.gz

4.安装步骤
4.1更改服务器名

hostnamectl set-hostname lvs01（3.105）
hostnamectl set-hostname lvs02 （3.104）
hostnamectl set-hostname rs01（3.107）
hostnamectl set-hostname rs02 （3.106）

4.2关闭防火墙和selinux

firewall-cmd --state
systemctl stop firewalld.service
systemctl disable firewalld.service
sed -i ‘s/^ *SELINUX=disabled/SELINUX=disabled/g‘ /etc/selinux/config
setenforce 0

4.3ipvs安装（192.168.3.105，192.168.3.104）

yum install ipvsadm( yum install --downloadonly ipvsadm（下载安装包)rpm -ivh ipvsadm*)
ipvsadm
lsmod | grep ip_vs

4.4 在2台 rs 上配置脚本（192.168.3.106,192.168.3.107）

#! /bin/bash
vip=192.168.3.114
ifconfig lo:0 $vip broadcast $vip netmask 255.255.255.255 up
route add -host $vip lo:0
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
在lvs两台上分别执行bash /usr/local/sbin/lvs_dr_rs.sh

4.5keepalived安装

yum -y install openssl openssl-devel  
yum -y install libnl libnl-devel
yum install -y libnfnetlink-devel

./configure --prefix=/usr/local/keepalived
make
make install

cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/keepalived
mkdir /etc/keepalived
cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
cp /usr/local/keepalived/etc/init.d/keepalived /etc/init.d/

4.6keepalived配置

192.168.3.105
vrrp_instance VI_1 {
    state MASTER
interface ens160
virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.3.114
    }
}

virtual_server 192.168.3.114 89 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    persistence_timeout 0
    protocol TCP

    real_server 192.168.3.106 89 {
        weight 1
        TCP_CHECK {
            connect_timeout 10
            nb_get_retry 3
            delay_before_retry 3
            connect_port 89
        }
    }

    real_server 192.168.3.107 89 {
        weight 1
        TCP_CHECK {
            connect_timeout 10
            nb_get_retry 3
            delay_before_retry 3
            connect_port 89
        }
    }
}
192.168.3.104
vrrp_instance VI_1 {
    state BACKUP
    interface ens160
    virtual_router_id 51
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.3.114
    }
}

virtual_server 192.168.3.114 89 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    persistence_timeout 0
    protocol TCP

    real_server 192.168.3.106 89 {
        weight 1
        TCP_CHECK {
            connect_timeout 10
            nb_get_retry 3
            delay_before_retry 3
            connect_port 89
        }
    }

    real_server 192.168.3.107 89 {
        weight 1
        TCP_CHECK {
            connect_timeout 10
            nb_get_retry 3
            delay_before_retry 3
            connect_port 89
        }
    }
}
keepalived的2个节点执行如下命令，开启转发功能
echo 1 > /proc/sys/net/ipv4/ip_forward
先主后从分别启动keepalive
service keepalived start

5.验证
5.1lvs负载均衡

手动关闭192.168.3.107的nginx
不再出现107

重新开启nginx
107又出现了

5.2 验证keepalived高可用

此时vip192.168.3.114在192.168.3.1105上

模拟宕机，关闭192.168.3.105的keepalived，192.168.3.114漂移到了备用节点192.168.3.104

仍然可以正常访问

6.参考
https://www.cnblogs.com/liwei0526vip/p/6370103.html