linux运维架构之路-keepalived高可用

Posted Demon

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了linux运维架构之路-keepalived高可用相关的知识,希望对你有一定的参考价值。

一、Keepalived介绍

         Keepalived起初是专为LVS负载均衡软件设计的,用来管理并监控LVS集群系统中各个服务节点的状态,后来又加入了可以实现高可用的VRRP功能,Keepalived是一款高可用软件,它的功能主要包括:

1、管理LVS负载均衡软件

2、实现对LVS集群节点健康检查功能

3、作为系统网络服务的高可用功能

二、Keepalived工作原理

Keepalived的实现基于VRRP

1、VRRP协议,全称Virtual Router Redundancy Protocol,中文名为虚拟路由冗余协议,VRRP的出现是为了解决静态路由的单点故障。
2、VRRP是用过IP多播的方式(默认多播地址(224.0.0.18))实现高可用对之间通信的。
3、工作时主节点发包,备节点接包,当备节点接收不到主节点发的数据包的时候,就启动接管程序接管主节点的资源。备节点可以有多个,通过优先级竞选,但一般Keepalived系统运维工作中都是一对

三、keepalived部署

1、安装keepalived(lb01 lb02)

rpm -qa keepalived
yum install keepalived -y

2、keepalived配置文件详解

global_defs {                                    --- 全局配置标题
   notification_email {                          --- 定义管理员邮箱信息,
     330882721@qq.com
     330442721@qq.com
   }
   notification_email_from oldboy@163.com        --- 定义利用什么邮箱发送邮件
   smtp_server smtp.163.com                      --- 定义邮件服务器信息
   smtp_connect_timeout 30                       --- 定义邮件发送超时时间
   router_id oldboy01                            --- (重点参数)局域网keepalived主机身份标识信息,每一个keepalived主机身份标识信息唯一
}

vrrp_instance VI_1 {                             --- vrrp协议相关配置(vip地址设置)
    state MASTER                                 --- keepalived角色描述(状态)信息,可以配置参数(MASTER BACKUP)
    interface eth0                               --- 表示将生成虚IP地址,设置在指定的网卡上
    virtual_router_id 51                         --- 表示keepalived家族标识信息
    priority 100                                 --- keepalived服务竞选主备服务器优先级设置(越大越优先)
    advert_int 1                                 --- 主服务组播包发送间隔时间       
    authentication {                             --- 主备主机之间通讯认证机制,
        auth_type PASS                           --- 采用明文认证机制
        auth_pass 1111                           --- 编写明文密码
    }
    virtual_ipaddress {                          --- 设置虚拟IP地址信息
        10.0.0.3
    }
}

3、搭建基础的keepalived配置文件

#lb01主

global_defs {

   router_id LVS_01

}

 

vrrp_instance VI_1 {

    state MASTER

    interface eth0

    virtual_router_id 51

    priority 150

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass 1111

    }

    virtual_ipaddress {

     10.0.0.3/24 dev eth0 label eth0:1

    }

}

虚拟IP地址显示信息:
默认显示信息:inet 10.0.0.3/32 scope global eth0
修改显示信息:inet 10.0.0.3/24 scope global secondary eth0:1

#lb02备

global_defs {

   router_id LVS_02

}

 

vrrp_instance VI_1 {

    state BACKUP

    interface eth0

    virtual_router_id 51

    priority 100

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass 1111

    }

    virtual_ipaddress {

     10.0.0.3/24 dev eth0 label eth0:1

    }

}

虚拟IP地址显示信息:
默认显示信息:inet 10.0.0.3/32 scope global eth0
修改显示信息:inet 10.0.0.3/24 scope global secondary eth0:1

测试说明:进行抓包观察配置效果;并且对比两个负载均衡服务器的配置文件

四、keepaliver

脑产生的原因

①高可用服务器之间心跳线链路发生故障,导致无法正常通信,心跳线坏了(包括断了,老化)

②网卡及相关驱动坏了,IP配置及冲突问题(网上直连)

③心跳线间连接的设置故障(网上及交换机)

④高可用服务器上开启了iptables防火墙阻挡了心跳消息传输

高可用服务器上心跳网卡地址等信息配置不正确,导致发送心跳失败

 解决裂脑常见方案

①同时使用串行电缆和以太网电缆连接,同时用两条心跳线路

②当检测裂脑时强行关闭一个心跳节点(这个功能需要特殊设备支持,如stonith、fence)

③运维层面做好对裂脑的监控报警

#制作监控脚本---lb02 
报警的条件:只要lb02 上面有vip
1.lb01 挂了
2.心碎

#!/bin/bash
#desc: jiankong lb02 vip 
if [ `ip a s eth0 |grep -c "10.0.0.3"` == 1 ];then
   echo "baojing"
fi

五、企业实践案例一:nginx反向代理只监听vip地址,防攻击

1、企业keepalived服务应用(修改nginx反向代理只监听vip地址)

#lb01 lb02 nginx配置
worker_processes 1; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; sendfile on; keepalive_timeout 65; log_format main \'$remote_addr - $remote_user [$time_local] "$request" \' \'$status $body_bytes_sent "$http_referer" \' \'"$http_user_agent" "$http_x_forwarded_for"\'; upstream server_pools { server 10.0.0.7; server 10.0.0.8; server 10.0.0.9; } server { listen 10.0.0.3:80; server_name www.etiantian.org; location / { proxy_pass http://server_pools; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $remote_addr; } access_log logs/access_www.log main; } server { listen 10.0.0.3:80; server_name blog.etiantian.org; location / { proxy_pass http://server_pools; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $remote_addr; } access_log logs/access_blog.log main; } }
说明:在修改反向代理服务器配置文件监听地址时,多个server都需要配置监听地址,否则仍旧使用默认监听所有,nginx修改ip相关的必须重启服务,平滑重启不启作用

2、lb02上不存在vip地址,无法监听,需要修改内核文件

解决方法:
echo \'net.ipv4.ip_nonlocal_bind = 1\' >>/etc/sysctl.conf
sysctl -p 

六、企业实践案例二:keepalived结合脚本监控nginx服务

1、nginx服务停止,keepalived服务自动停止,vip飘走

 #!/bin/bash
 #name: check_web.sh
 #desc: check nginx and kill keepalived 
 if [ `ps -ef |grep nginx |grep -v grep |wc -l` -lt 2  ];then
     /etc/init.d/keepalived stop 
 fi

2、把监控脚本放入keepalived配置文件中

 global_defs {
   router_id LVS_02
}

vrrp_script check_web {
script "/server/scripts/check_web.sh"    --- 表示将一个脚本信息赋值给变量check_web
interval 2                               --- 执行监控脚本的间隔时间
weight 2                                 --- 利用权重值和优先级进行运算,从而降低主服务优先级
                                             使之变为备服务器(建议先忽略)
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
     10.0.0.3/24 dev eth0 label eth0:1
    }
  track_script {
         check_web
  }
}

七、企业实践案例三:keepalived多实例配置

#lb01
global_defs { router_id LVS_01 } vrrp_instance VI_1 { state MASTER
interface eth0 virtual_router_id 51 priority 150 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 10.0.0.3/24 dev eth0 label eth0:1 } } vrrp_instance VI_2 { state BACKUP interface eth0 virtual_router_id 52 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 10.0.0.4/24 dev eth0 label eth0:2 } }

#lb02 
global_defs {
   router_id LVS_02
}
vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
     10.0.0.3/24 dev eth0 label eth0:1
    }
}
vrrp_instance VI_2 {
    state MASTER
    interface eth0
    virtual_router_id 52
    priority 150 
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }   
    virtual_ipaddress {
     10.0.0.4/24 dev eth0 label eth0:2
    }   

}
#lb01 lb02  nginx.conf 
worker_processes  1;
events {
    worker_connections  1024;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;
    log_format  main  \'$remote_addr - $remote_user [$time_local] "$request" \'
                      \'$status $body_bytes_sent "$http_referer" \'
                      \'"$http_user_agent" "$http_x_forwarded_for"\';              
    upstream server_pools {
        server 10.0.0.7;
        server 10.0.0.8;
        server 10.0.0.9;
    }
    server {
        listen 10.0.0.3:80;
        server_name www.etiantian.org;
        location / {
            proxy_pass http://server_pools;
            proxy_set_header Host $host;
            proxy_set_header X-Forwarded-For $remote_addr;
        }
        access_log  logs/access_www.log  main;
    }
        server {
        listen 10.0.0.4:80;
        server_name blog.etiantian.org;
        location / {
            proxy_pass http://server_pools;
            proxy_set_header Host $host;
            proxy_set_header X-Forwarded-For $remote_addr;
        }
        access_log  logs/access_blog.log  main;
        
    }
}

windows hosts解析

10.0.0.3  www.etiantian.org
10.0.0.4  bbs.etiantian.org

八、指定文件接收Keepalived服务日志

vi /etc/sysconfig/keepalived
KEEPALIVED_OPTIONS="-D -S 0 -d"

vi /etc/rsyslog.conf
local0.*                                                /var/log/keepalived.log
/etc/init.d/keepalived restart

查看生成的接收keepalived日志文件

ll /var/log/keepalived.log
-rw------- 1 root root 5600 Oct 13 11:43 /var/log/keepalived.log

 

以上是关于linux运维架构之路-keepalived高可用的主要内容,如果未能解决你的问题,请参考以下文章

Linux运维:测试HAProxy+Keepalived高可用负载均衡集群

Linux运维学习之keepalived实现高可用集群

Linux运维:搭建HAProxy+Keepalived高可用负载均衡系统

linux运维架构之路-redis集群

HAProxy基于KeepAlived实现Web高可用及动静分离

Linux运维:通过MMM构建MySQL高可用集群系统