关于Linux下HAProxy自动化部署的一些笔记整理

Posted 2022-12-30 山河已无恙

写在前面

---

• 学习 OKD ，里面的路由组件通过 HAproxy 实现
• 所以这里学习 HAproxy，博文内容涉及
• 手工部署 HAproxy
• 编写 Ansible 角色自动部署 HAproxy
• 理解不足小伙伴帮忙指正

傍晚时分，你坐在屋檐下，看着天慢慢地黑下去，心里寂寞而凄凉，感到自己的生命被剥夺了。当时我是个年轻人，但我害怕这样生活下去，衰老下去。在我看来，这是比死亡更可怕的事。--------王小波

---

HAProxy 是一种免费、非常快速且可靠的反向代理，可为基于 TCP 和 HTTP 的应用程序提供 高可用性、 负载平衡和代理。它特别适用于流量非常大的网站，并为世界上访问量最大的网站中的很大一部分提供支持。多年来，它已成为事实上的标准开源负载均衡器，现在随大多数主流 Linux 发行版一起提供，并且通常默认部署在云平台中。

HAProxy 核心团队并行维护多个版本。从1.8版本开始，每年发布两个主要版本。第一个数字通常表示重大更改(配置格式等)，但实际上很少更改。第二位数字表示新功能。两者构成一个分支。这些数字后面会出现一个额外的数字，表示错误修复版本。

对应负载均衡来讲性能对比： LVS > HAProxy > nginx ，一般情况下 HAProxy和Nginx 做七层代理，因为可以会话保持，LVS 做四层的负载均衡.

手动安装配置

┌──[[e-mail]]-[~]
└─$rpm -ql haproxy || yum -y install yum -y install haproxy18.x86_64

安装版本

┌──[[e-mail]]-[~]
└─$haproxy18 -v
HA-Proxy version 1.8.27-493ce0b 2020/11/06
Copyright 2000-2020 Willy Tarreau <[e-mail]>rpm 

安装之后我们来分析一下配置文件,配置注释写的很清楚，完整的配置文件配置我们可以参考 http://haproxy.1wt.eu/download/1.8/doc/configuration.txt

作为一个代理服务器，配置相对简单。

┌──[[e-mail]]-[~]
└─$rpm -qc haproxy18
/etc/haproxy18/haproxy.cfg
/etc/logrotate.d/haproxy18
/etc/sysconfig/haproxy18

修改配置文件，配置文件默认有一个配置 Demo。直接修改就可以，同时默认做了动静分离的配置，不同的版本配置文件略有差异，所以要以对应的配置 Demo 为准，

┌──[[e-mail]]-[~]
└─$cat /etc/haproxy18/haproxy.cfg
#---------------------------------------------------------------------
# Example configuration for a possible web application.  See the
# full configuration options online.
#
#   https://www.haproxy.org/download/1.8/doc/configuration.txt
#
#---------------------------------------------------------------------

#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
    # to have these messages end up in /var/log/haproxy.log you will
    # need to:
    #
    # 1) configure syslog to accept network log events.  This is done
    #    by adding the '-r' option to the SYSLOGD_OPTIONS in
    #    /etc/sysconfig/syslog
    #
    # 2) configure local2 events to go to the /var/log/haproxy.log
    #   file. A line like the following can be added to
    #   /etc/sysconfig/syslog
    #
    #    local2.*                       /var/log/haproxy.log
    #
    log         127.0.0.1 local2

    chroot      /var/lib/haproxy18
    pidfile     /var/run/haproxy18.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon

    # turn on stats unix socket
    stats socket /var/lib/haproxy18/stats

#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
    mode                    http
    log                     global
    option                  httplog
    option                  dontlognull
    option http-server-close
    option forwardfor       except 127.0.0.0/8
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 3000

#---------------------------------------------------------------------
# main frontend which proxys to the backends
#---------------------------------------------------------------------
frontend main
    bind *:5000
    acl url_static       path_beg       -i /static /images /javascript /stylesheets
    acl url_static       path_end       -i .jpg .gif .png .css .js

    use_backend static          if url_static
    default_backend             app

#---------------------------------------------------------------------
# static backend for serving up images, stylesheets and such
#---------------------------------------------------------------------
backend static
    balance     roundrobin
    server      static 127.0.0.1:4331 check

#---------------------------------------------------------------------
# round robin balancing between the various backends
#---------------------------------------------------------------------
backend app
    balance     roundrobin #  可以的负载方式 static-rr，rdp-cookie，hdr, url_param ,url,  source,leastconn   
    server  app1 192.168.29.152:5001 check 
    server  app2 192.168.26.152:5002 check

#server 关键字，指服务器
#web1，web2 为服务器名，可以和主机名相同，也可以不相同
#192.168.2.100:80  真实后端服务器的地址和访问端口
#check  健康检查的关键字
listen haproxystats
    bind  *:5009
    stats refresh 30s
    stats uri /stats
    stats realm Haproxy Manager
    stats auth admin:admin    

这里在原来的配置文件中加了监控页面。启动服务

┌──[[e-mail]]-[~]
└─$systemctl restart haproxy18.service

然后我们需要做一些准备工作。对应的负载机器使用 python 自带的 httpServer 模块发布服务

192.168.26.152:5002

┌──[[e-mail]]-[~/vms152.liruilongs.github.io]
└─$touch 192.168.26.152:5002
┌──[[e-mail]]-[~/vms152.liruilongs.github.io]
└─$coproc  python -m SimpleHTTPServer 5002
[1] 22913

192.168.29.152:5001

[root@vms152 vm15]# touch 192.168.29.152:5001
[root@vms152 vm15]# coproc python -m SimpleHTTPServer 5001
[1] 14384

当前主机提供静态文件 192.168.26.154：4331

┌──[[e-mail]]-[~]
└─$coproc  python -m SimpleHTTPServer 4331
[1] 9841

访问测试，默认为加权轮询

┌──[[e-mail]]-[~]
└─$ curl 192.168.26.154:5000 -s | grep '<li>'
<li><a href="192.168.26.152%3A5002">192.168.26.152:5002</a>
┌──[[e-mail]]-[~]
└─$ curl 192.168.26.154:5000 -s | grep '<li>'
<li><a href="192.168.29.152%3A5001">192.168.29.152:5001</a>
┌──[[e-mail]]-[~]
└─$

健康检查,把 192.168.26.152:5002 的服务干掉

┌──[[e-mail]]-[~/vms152.liruilongs.github.io]
└─$kill %1

负载全部到了 192.168.29.155:5001

┌──[[e-mail]]-[~]
└─$ curl 192.168.26.154:5000 -s | grep '<li>'
<li><a href="192.168.29.152%3A5001">192.168.29.152:5001</a>
┌──[[e-mail]]-[~]
└─$ curl 192.168.26.154:5000 -s | grep '<li>'
<li><a href="192.168.29.152%3A5001">192.168.29.152:5001</a>
┌──[[e-mail]]-[~]
└─$ curl 192.168.26.154:5000 -s | grep '<li>'
<li><a href="192.168.29.152%3A5001">192.168.29.152:5001</a>
┌──[[e-mail]]-[~]
└─$

监控页面查看

Ansible自动化部署角色编写

安装比较简单，没有其他的逻辑，所以角色编写简单，这里如果涉及防火墙之类，需要添加角色依赖

角色创建

┌──[[e-mail]]-[~/ansible]
└─$ansible-galaxy init role-haproxy18 --init-path ./roles/
- Role role-haproxy18 was created successfully
┌──[[e-mail]]-[~/ansible]
└─$ansible-galaxy list | grep ha
- role-haproxy18, (unknown version)

任务执行,访问测试

┌──[[e-mail]]-[~/ansible]
└─$echo "192.168.26.154" > haproxy_list
┌──[[e-mail]]-[~/ansible]
└─$ansible-playbook deploy_haproxy.yaml -i haproxy_list
┌──[[e-mail]]-[~/ansible]
└─$curl 192.168.26.154:5000 -s | grep /a
<li><a href="192.168.29.152%3A5001">192.168.29.152:5001</a>

角色任务执行的剧本

┌──[[e-mail]]-[~/ansible]
└─$cat deploy_haproxy.yaml
- name: Ensure HAProxy is deployed
  hosts: all
  force_handlers: True

  roles:
    - role: role-haproxy18
      haproxy_port: 5000
      haproxy_appservers:
      - name: 192.168.26.152
        ip: 192.168.26.152
        backend_port: 5002
      - name: 192.168.29.152
        ip: 192.168.29.152
        backend_port: 5001

主任务文件

┌──[[e-mail]]-[~/ansible]
└─$cat roles/role-haproxy18/tasks/main.yml
---
# tasks file for role-haproxy18

- name: config aliyun repo
  yum_repository:
    name: aliyunrepo
    description: aliyun repo
    baseurl: http://mirrors.aliyun.com/centos/$releasever/os/$basearch/
    gpgcheck: no


# tasks file for haproxydd
- name: Ensure haproxy packages are present
  yum:
    name:
      - haproxy18
      - socat
    state: present

- name: Ensure haproxy is started and enabled
  service:
    name: haproxy18
    state: started
    enabled: yes

- name: Ensure haproxy configuration is set
  template:
    src: haproxy.cfg.j2
    dest: /etc/haproxy18/haproxy.cfg
  notify: reload haproxy

处理器文件

┌──[[e-mail]]-[~/ansible]
└─$cat roles/role-haproxy18/handlers/main.yml
---
# handlers file for role-haproxy18

- name: restart haproxy
  service:
    name: haproxy18
    state: restarted

- name: reload haproxy
  service:
    name: haproxy18
    state: reloaded

默认变量

┌──[[e-mail]]-[~/ansible]
└─$cat roles/role-haproxy18/defaults/main.yml
---
# defaults file for role-haproxy18
# Log-level for HAProxy logs
haproxy_log_level: info

# Port exposed to clients
haproxy_port: 80

# Name for the default backend
haproxy_backend_name: app

# Port backend is exposed to
haproxy_backend_port: 80

# The appservers variable is a list
# of backend servers that provide
# the web service that is proxied
# haproxy.  Each server must define:
# name, address, port. Below is
# and example structure:
# haproxy_appservers: []
#   - name: serverb.lab.example.com
#     ip_address: 1.2.3.4
#     port: 5000
#   - name: serverc.lab.example.com
#     ip_address: 1.2.3.5
#     port: 5000
# The default is no defined backend servers.
haproxy_appservers: []

# Haproxy Manager
haproxy_stats_port: 29999
stats_url: /stats
stats_user: admin
stats_pass: admin

# Socket used to communicate with haproxy service. DO NOT CHANGE
haproxy_socket: /var/run/haproxy.sock

配置文件模板文件

┌──[[e-mail]]-[~/ansible]
└─$cat roles/role-haproxy18/templates/haproxy.cfg.j2
#---------------------------------------------------------------------
# Example configuration for a possible web application.  See the
# full configuration options online.
#
#   https://www.haproxy.org/download/1.8/doc/configuration.txt
#
#---------------------------------------------------------------------

#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
    # to have these messages end up in /var/log/haproxy.log you will
    # need to:
    #
    # 1) configure syslog to accept network log events.  This is done
    #    by adding the '-r' option to the SYSLOGD_OPTIONS in
    #    /etc/sysconfig/syslog
    #
    # 2) configure local2 events to go to the /var/log/haproxy.log
    #   file. A line like the following can be added to
    #   /etc/sysconfig/syslog
    #
    #    local2.*                       /var/log/haproxy.log
    #
    log         127.0.0.1 local2  haproxy_log_level 

    chroot      /var/lib/haproxy18
    pidfile     /var/run/haproxy18.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon

    # turn on stats unix socket
    stats socket  haproxy_socket  level admin

#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
    mode                    http
    log                     global
    option                  httplog
    option                  dontlognull
    option http-server-close
    option forwardfor       except 127.0.0.0/8
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 3000

#---------------------------------------------------------------------
# main frontend which proxys to the backends
#---------------------------------------------------------------------
frontend main
    bind *: haproxy_port 
  # acl url_static       path_beg       -i /static /images /javascript /stylesheets
  # acl url_static       path_end       -i .jpg .gif .png .css .js

   # use_backend static          if url_static
    default_backend               haproxy_backend_name 

#---------------------------------------------------------------------
# static backend for serving up images, stylesheets and such
#---------------------------------------------------------------------
#backend static
   # balance     roundrobin
   # server      static 127.0.0.1:4331 check

#---------------------------------------------------------------------
# round robin balancing between the various backends
#---------------------------------------------------------------------

backend  haproxy_backend_name 
    balance     roundrobin
% for server in haproxy_appservers %
    server  server.name   server.ip : server.backend_port  check
% endfor %


listen haproxystats
    bind  *: haproxy_stats_port 
    stats refresh 30s
    stats uri  stats_url  
    stats realm Haproxy Manager
    stats auth  stats_user : stats_pass 


┌──[[e-mail]]-[~/ansible]
└─$

博文参考

---

https://www.haproxy.org/