Keepalived+HAProxy实现K8S高可用集群-20210927

Posted 我也想做架构师啊

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了Keepalived+HAProxy实现K8S高可用集群-20210927相关的知识,希望对你有一定的参考价值。

本文采用Keepalived+HAProxy的方式构建高可用集群。
当你如果你有硬件负载均衡设备当然更好了。

准备环境:

主机ip
k8s-master01192.168.10.4
k8s-master02192.168.10.5
k8s-master03192.168.10.6
VIP192.168.10.150
  • 架构图

注意:master集群采用奇数台数,3、5、7…

  • 所有节点都进行hosts文件解析
tail -3 /etc/hosts
192.168.10.4 k8s-master01
192.168.10.5 k8s-master02
192.168.10.6 k8s-master03
  • 所有节点都要安装keepalived和haproxy软件
yum -y install haproxy keepalived
  • 修改haproxy配置文件(所有节点配置相同)
  • 最好选择2.x版本,当然这个版本也不影响使用,只是功能没有2.x版本多
vim /etc/haproxy/haproxy.cfg
global
  maxconn  2000
  ulimit-n  16384
  log  127.0.0.1 local0 err
  stats timeout 30s

defaults
  log global
  mode  http
  option  httplog
  timeout connect 5000
  timeout client  50000
  timeout server  50000
  timeout http-request 15s
  timeout http-keep-alive 15s

frontend monitor-in
  bind *:33305
  mode http
  option httplog
  monitor-uri /monitor

listen stats
  bind    *:8006
  mode    http
  stats   enable
  stats   hide-version
  stats   uri       /stats
  stats   refresh   30s
  stats   realm     Haproxy\\ Statistics
  stats   auth      admin:admin

frontend k8s-master
  bind 0.0.0.0:16443
  bind 127.0.0.1:16443
  mode tcp
  option tcplog
  tcp-request inspect-delay 5s
  default_backend k8s-master

backend k8s-master
  mode tcp
  option tcplog
  option tcp-check
  balance roundrobin
  default-server inter 10s downinter 5s rise 2 fall 2 slowstart 60s maxconn 250 maxqueue 256 weight 100
  server k8s-master01	192.168.10.4:6443  check
  server k8s-master02	192.168.10.5:6443  check
  server k8s-master03   192.168.10.6:6443  check
  • master01节点修改keepalived配置文件
vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
    router_id LVS_DEVEL
}
vrrp_script chk_apiserver {
    script "/etc/keepalived/check_apiserver.sh"
    interval 2
    weight -5
    fall 3  
    rise 2
}
vrrp_instance VI_1 {
    state MASTER
    interface eth0
    mcast_src_ip 192.168.10.4
    virtual_router_id 51
    priority 100
    advert_int 2
    authentication {
        auth_type PASS
        auth_pass K8SHA_KA_AUTH
    }
    virtual_ipaddress {
        192.168.10.150/24
    }
    track_script {
       chk_apiserver
    }
  • master02节点修改keepalived配置文件
vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
    router_id LVS_DEVEL
}
vrrp_script chk_apiserver {
    script "/etc/keepalived/check_apiserver.sh"
    interval 2
    weight -5
    fall 3  
    rise 2
}
vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    mcast_src_ip 192.168.10.5
    virtual_router_id 51
    priority 50
    advert_int 2
    authentication {
        auth_type PASS
        auth_pass K8SHA_KA_AUTH
    }
    virtual_ipaddress {
        192.168.10.150/24
    }
    track_script {
       chk_apiserver
    }
}
  • master03节点修改keepalived配置文件
vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
    router_id LVS_DEVEL
}
vrrp_script chk_apiserver {
    script "/etc/keepalived/check_apiserver.sh"
    interval 2
    weight -5
    fall 3  
    rise 2
}
vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    mcast_src_ip 192.168.10.6
    virtual_router_id 51
    priority 50
    advert_int 2
    authentication {
        auth_type PASS
        auth_pass K8SHA_KA_AUTH
    }
    virtual_ipaddress {
        192.168.10.150/24
    }
    track_script {
       chk_apiserver
    }
}
  • 所有节点创建健康检查脚本
vim /etc/keepalived/check_apiserver.sh
#!/bin/bash

err=0
for k in $(seq 1 5)
do
    check_code=$(pgrep haproxy)
    if [[ $check_code == "" ]]; then
        err=$(expr $err + 1)
        sleep 5
        continue
    else
        err=0
        break
    fi
done

if [[ $err != "0" ]]; then
    echo "systemctl stop keepalived"
    /usr/bin/systemctl stop keepalived
    exit 1
else
    exit 0
fi
  • 启动haproxy与keepalived服务
systemctl daemon-reload 
systemctl enable --now haproxy
systemctl enable --now keepalived
  • 可以用ping和telnet命令测试一下vip的可用性
ping 192.168.10.150
PING 192.168.10.150 (192.168.10.150) 56(84) bytes of data.
64 bytes from 192.168.10.150: icmp_seq=1 ttl=64 time=1.60 ms
64 bytes from 192.168.10.150: icmp_seq=2 ttl=64 time=0.519 ms
64 bytes from 192.168.10.150: icmp_seq=3 ttl=64 time=0.874 ms
64 bytes from 192.168.10.150: icmp_seq=4 ttl=64 time=0.786 ms
^C
--- 192.168.10.150 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3009ms
rtt min/avg/max/mdev = 0.519/0.946/1.606/0.403 ms

telnet 192.168.10.150 16443
Trying 192.168.10.150...
Connected to 192.168.10.150.
Escape character is '^]'.
Connection closed by foreign host.

  • 再尝试一下断开vip所在节点的keepalived,看ip是否漂移,如果vip漂移至另一节点则代表成功

  • 可能难免有地方出错,如果出错可以留言哈

以上是关于Keepalived+HAProxy实现K8S高可用集群-20210927的主要内容,如果未能解决你的问题,请参考以下文章

k8s高可用部署后续:SLB

k8s高可用部署后续:SLB

k8s高可用部署:keepalived + haproxy

keepalived高可用

第三篇(二进制部署k8s集群---Flannel网络和keepalived+haproxy高可用)

HAproxy+Keepalived实现高可用