Centos Keepalived-Ha集群+Lvs-DR+Nginx集群配置

Posted

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了Centos Keepalived-Ha集群+Lvs-DR+Nginx集群配置相关的知识,希望对你有一定的参考价值。

背景介绍 

  我们都知道一台服务器能提供的资源是有限的,如果想打破这个限制,只有两个办法,一个是购买硬件,提升服务器性能,但是性价比过高,而第二种方法这则是,使用集群软件让两台服务器处理一个服务的不同请求,分带单台服务器的负载,从而提升性能,实现这一的软件开源著名的有nginx或者Lvs,这里我们使用的是LVS,但是lvs需要一个调度服务器,这样调度服务器就是一个集训的单点故障,一旦调度服务器出现故障,整个集群便崩溃,所以我们需要两台调度节点,一旦一个调度节点损坏,那么另一台调度节点开始工作,这里我们使用Keepalived开源软件实现调度器的高可用。

技术分享

逻辑图介绍

    1.用户在访问Nginx的时候,先通过VIP访问Master1,由Master指定那个real-server提供服务

    2.在Master通过lvs-Dr指定的算法选定由Nginx1来提供服务,那么数据在通过VIP发送到Nginx1上

    3.Nginx处理完请求后,在通过自身的VIP将处理后的请求直接发送给User

    注明:

        如果Master出现故障,那么Baskup将替代Master工作

        如果Nginx1出现故障,那么Nginx2将提点Nginx1工作

        如果Nginx1没有出现故障,那么LVS服务使用算法会将请求平均到Nginx1和Nginx2上

        如果两台Nginx同时出现故障,那么Master会提供Sorry Server服务

技术分享

物理拓扑图及环境介绍

    服务器:使用虚拟机虚拟4套服务器

    操作系统:CentOS7.3

    HA软件/LVs软件:keepalived

    Web软件:Nginx


IP及主机名配置

    VIP:172.18.17.30

    Master:

        主机名:shiyan7.one

        IP地址:172.18.17.31

    Backup

        主机名:shiyan7.fore

        IP地址:172.18.17.34

     Nginx1

        主机名:shiyan7.two

        IP地址:172.18.17.34

     Nginx2

        主机名:shiyan7.thy

        IP地址:172.18.17.34

   


4台服务器初始化配置

    同步时间服务

    #Master配置NTP服务器
     [[email protected] ~ ]# vim /etc/ntp.conf
       restrict 172.18.17.0 mask 255.255.0.0 nomodify notrap
       server 127.127.1.0
       fudge 127.1.1.0 stratum 8
    [[email protected] ~ ]# ntpdate 172.18.17.31

    关闭Selinux

     [[email protected] ~ ]# vim /etc/selinux/config
       SELINUX=disabled

    关闭防火墙

     [[email protected] ~ ]# systemctl stop firewalld
     [[email protected] ~ ]# systemctl disable firewalld
      [[email protected] ~ ]# iptables -F

    主机名解析配置

     

      [[email protected] ~ ]# cat /etc/hosts
            127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
            ::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
            172.18.17.34 shiyan.fore
            172.18.17.31 shiyan.one
            172.18.17.32 shiyan7.two
            172.18.17.33 shiyan7.thy

Master/Backup初始化配置

    安装KeepAlived、Nginx软件

        

      [[email protected] ~ ]# yum install keepalived nginx ipvsadm

Nginx1/2初始化配置

    安装Nginx软件

      [[email protected] ~ ]# yum install nginx
      #这里配置一个配置VIP及更改内核参数的脚本。
            [[email protected] ~ ]# touch roush.sh
            [[email protected] ~ ]# chmod +x roush.sh
            [[email protected] ~ ]# vim roush.sh
                #!/bin/bash    
                #
                
                vip="172.18.17.30"
                netmask="255.255.255.255"
                iface="lo:0"
                
                case $1 in
                start)
                        echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
                        echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
                        echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
                        echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
                        ifconfig $iface $vip netmask $netmask broadcast $vip up
                        route add -host $vip dev $iface
                        ;;
                stop)
                        ifconfig $iface down
                        echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
                        echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
                        echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
                        echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
                        ;;
                esac


Ngxin1配置

     [[email protected] ~ ]# ./roush.sh start
    [[email protected] ~ ]# mkdir /app/nginx/html
    [[email protected] ~ ]#  echo "<h1>RS1:172.18.17.32</h1>" > /app/nginx/html/index.html
    [[email protected] ~ ]#  cp /etc/nginx/nginx/conf{,.bak}#备份原文件
    [[email protected] ~ ]#  vim /etc/nginx/nginx/conf
            server {
            listen       80 default_server;
            listen       [::]:80 default_server;
            server_name  _;
            root         /app/nginx/html;#修改root这里的路径,其他不需要
            # Load configuration files for the default server block.
            include /etc/nginx/default.d/*.conf;
    
            location / {
            }
    
            error_page 404 /404.html;
                location = /40x.html {
            }
    
            error_page 500 502 503 504 /50x.html;
                
            }
    [[email protected] ~ ]# systemctl start nginx
    #测试Nginx服务
    [[email protected] ~ ]# curl http://172.18.17.33
    <h1>RS1:172.18.17.33</h1>


Ngxin2配置

     [[email protected] ~ ]# ./roush.sh start
    [[email protected] ~ ]# mkdir /app/nginx/html
    [[email protected] ~ ]#  echo "<h1>RS1:172.18.17.33</h1>" > /app/nginx/html/index.html
    [[email protected] ~ ]#  cp /etc/nginx/nginx/conf{,.bak}#备份原文件
    [[email protected] ~ ]#  vim /etc/nginx/nginx/conf
            server {
            listen       80 default_server;
            listen       [::]:80 default_server;
            server_name  _;
            root         /app/nginx/html;#修改root这里的路径,其他不需要
            # Load configuration files for the default server block.
            include /etc/nginx/default.d/*.conf;
    
            location / {
            }
    
            error_page 404 /404.html;
                location = /40x.html {
            }
    
            error_page 500 502 503 504 /50x.html;
                
            }
    [[email protected] ~ ]# systemctl start nginx
    #测试Nginx服务
    [[email protected] ~ ]# curl http://172.18.17.34
    <h1>RS1:172.18.17.34</h1>



Master上的配置

    #配置Nginx1-Sorry Server
    [[email protected] ~ ]# mkdir /app/nginx/html
    [[email protected] ~ ]#  echo "Sorry server-1" > /app/nginx/html/index.html
    [[email protected] ~ ]#  cp /etc/nginx/nginx/conf{,.bak}#备份原文件
    [[email protected] ~ ]#  vim /etc/nginx/nginx/conf
        server {
            listen       80 default_server;
            listen       [::]:80 default_server;
            server_name  _;
            root         /app/nginx/html;#修改root这里的路径,其他不需要
            # Load configuration files for the default server block.
            include /etc/nginx/default.d/*.conf;
    
            location / {
            }
    
            error_page 404 /404.html;
                location = /40x.html {
            }
    
            error_page 500 502 503 504 /50x.html;
                
            }
    [[email protected] ~ ]# systemctl start nginx
    #测试Nginx1-Sorry Server
    [[email protected] ~ ]# curl http://172.18.17.31
        Sorry server-1
        
    #配置Keepalived服务
    [[email protected] ~ ]# cp /etc/keepalived/keepalived.conf{,.bak}
    [[email protected] ~ ]# vim /etc/keepalived/keepalived.conf
            vrrp_instance Ha_1 {
            state MASTER
            interface ens33
            virtual_router_id 51
            priority 100
            advert_int 1
            authentication {
                auth_type PASS
                auth_pass 1111
            }
            virtual_ipaddress {
                172.18.17.30/16 dev ens33
            }
        }
        
        virtual_server 172.18.17.30 80 {
            delay_loop 6
            lb_algo lc
            lb_kind DR
            persistence_timeout 50
            protocol TCP
            sorry_server 172.18.17.31 80
            
            real_server 172.18.17.32 80 {
               weight 1
                HTTP_GET {
                    url {
                      path /
                        status_code 200
                    }
                    connect_timeout 3
                    nb_get_retry 3
                    delay_before_retry 3
                }
        }
            real_server 172.18.17.33 80 {
                weight 1
                HTTP_GET {
                    url {
                      path /
                        status_code 200
                    }
                    connect_timeout 3
                    nb_get_retry 3
                    delay_before_retry 3
                }
            }
        }
    [[email protected] ~ ]#systemctl start keepalived
    #查看配置是否成功
    [[email protected] ~ ]# ip a l
        2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
        link/ether 00:0c:29:1b:f5:ae brd ff:ff:ff:ff:ff:ff
        inet 172.18.17.31/16 brd 172.18.255.255 scope global ens33
           valid_lft forever preferred_lft forever
        inet 172.18.17.30/16 scope global secondary ens33
           valid_lft forever preferred_lft forever
        [[email protected] ~ ]# ipvsadm -ln
            IP Virtual Server version 1.2.1 (size=4096)
            Prot LocalAddress:Port Scheduler Flags
              -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
            TCP  172.18.17.30:80 lc persistent 50
              -> 172.18.17.32:80              Route   1      0          0         
              -> 172.18.17.33:80              Route   1      0          0

Backup上的配置

    #配置Nginx1-Sorry Server
    [[email protected] ~ ]# mkdir /app/nginx/html
    [[email protected] ~ ]#  echo "Sorry server-2" > /app/nginx/html/index.html
    [[email protected] ~ ]#  cp /etc/nginx/nginx/conf{,.bak}#备份原文件
    [[email protected] ~ ]#  vim /etc/nginx/nginx/conf
        server {
            listen       80 default_server;
            listen       [::]:80 default_server;
            server_name  _;
            root         /app/nginx/html;#修改root这里的路径,其他不需要
            # Load configuration files for the default server block.
            include /etc/nginx/default.d/*.conf;
    
            location / {
            }
    
            error_page 404 /404.html;
                location = /40x.html {
            }
    
            error_page 500 502 503 504 /50x.html;
                
            }
    [[email protected] ~ ]# systemctl start nginx
    #测试Nginx1-Sorry Server
    [[email protected] ~ ]# curl http://172.18.17.34
        Sorry server-2
        
    #配置Keepalived服务
    [[email protected] ~ ]# cp /etc/keepalived/keepalived.conf{,.bak}
    [[email protected] ~ ]# vim /etc/keepalived/keepalived.conf
            vrrp_instance Ha_1 {
            state MASTER
            interface ens33
            virtual_router_id 51
            priority 100
            advert_int 1
            authentication {
                auth_type PASS
                auth_pass 1111
            }
            virtual_ipaddress {
                172.18.17.30/16 dev ens33
            }
        }
        
        virtual_server 172.18.17.30 80 {
            delay_loop 6
            lb_algo lc
            lb_kind DR
            persistence_timeout 50
            protocol TCP
            sorry_server 172.18.17.32 80
            
            real_server 172.18.17.32 80 {
               weight 1
                HTTP_GET {
                    url {
                      path /
                        status_code 200
                    }
                    connect_timeout 3
                    nb_get_retry 3
                    delay_before_retry 3
                }
        }
            real_server 172.18.17.33 80 {
                weight 1
                HTTP_GET {
                    url {
                      path /
                        status_code 200
                    }
                    connect_timeout 3
                    nb_get_retry 3
                    delay_before_retry 3
                }
            }
        }
    [[email protected] ~ ]#systemctl start keepalived
        [[email protected] ~ ]# ip a l
                2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
            link/ether 00:0c:29:1b:f5:ae brd ff:ff:ff:ff:ff:ff
            inet 172.18.17.34/16 brd 172.18.255.255 scope global ens33
               valid_lft forever preferred_lft forever
               #这里是Baskup服务所以不会有VIP地址
        [[email protected] ~ ]# ipvsadm -ln
            IP Virtual Server version 1.2.1 (size=4096)
            Prot LocalAddress:Port Scheduler Flags
              -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
            TCP  172.18.17.30:80 lc persistent 50
              -> 172.18.17.32:80              Route   1      0          0         
              -> 172.18.17.33:80              Route   1      0          0



测试服务

#正常测试
[[email protected] ~ ]# curl http://172.18.17.30
    <h1>RS1:172.18.17.32</h1>
    
[[email protected] ~ ]# curl http://172.18.17.30
    <h1>RS1:172.18.17.34</h1>
    
#关闭Nginx1测试
[[email protected] ~ ]# curl http://172.18.17.30
    <h1>RS1:172.18.17.33</h1>
    
#关闭所有Nginx服务测试
[[email protected] ~ ]# curl http://172.18.17.30
Sorry server-1

#关闭Master测试
[[email protected] ~ ]# curl http://172.18.17.30
Sorry server-2


本文出自 “tianrandai” 博客,请务必保留此出处http://tianrandai01.blog.51cto.com/12612752/1925874

以上是关于Centos Keepalived-Ha集群+Lvs-DR+Nginx集群配置的主要内容,如果未能解决你的问题,请参考以下文章

centos删除/home所在的lv后如何恢复

centos7.6 磁盘分区及扩容vg lv

Centos_Lvm_Create pv vg lv and mount

原Centos 7 下创建LVM流程

浅谈centos7的lvm(pv,vg,lv)

CentOS6虚拟机扩展磁盘空间—lvm