Haproxy

Posted 2023-04-01 K8S+ceph离线部署

Haproxy

负责均衡的作用

• Web服务器的动态水平扩展--->对用户无感知
• 增加业务并发访问及处理能力--->解决单服务器瓶颈问题
• 节约公网IP地址--->降低IT支持成本
• 隐藏内部服务器IP--->提高内部服务器安全性
• 配置简单--->固定格式的配置文件
• 功能丰富--->支持四层和七层，支持动态下线主机（传输层：四层。应用层：七层）
• 性能较强--->并发数万甚至数十万

负载均衡类型

与lvs的区别

• lvs是工作在内核级别的，看不到端口，而haproxy是工作的应用层级别的
• 四层
• LVS: Liunx Virtual Server
• Nginx: 1.9之后
• HaProxy: High Availability Proxy
• 七层
• HAProxy
• Nginx

应用场景

• 四层：Redis、Mysql、RabbitMQ、Memcache等
• 七层：Nginx、Tomcat、Apache、PHP、图片、动静分离、API等

安装与部署

• 企业版：https://www.haproxy.com/
• 社区版：http://www.haproxy.org/
• github：https://github.com/haproxy

• 内网IP地址划分

# 外部网络
172.16.0.0/16

# 内部网络
10.0.0.0/24

# 可用地址范围
10.0.0.1--10.0.0.254

yum安装

dnf install haproxy -y

rpm -ql haproxy

编译安装

解决lua环境

• Lua官网：www.lua.org
• Lua应用场景
• 游戏开发
• 独立应用开发
• Web应用脚本
• 扩展和数据库插件，如MySQL Proxy
• 安全系统，如入侵检测系统

# 当前系统版本
lua -v

# 安装基础命令及编译依赖环境
yum install -y gcc readline-devel

wget https://www.lua.org/ftp/lua-5.3.6.tar.gz

tar xvf lua-5.3.6.tar.gz -C /usr/local/src

cd /usr/local/src/lua-5.3.5

make linux test

# 查看编译安装的版本
src/lua -v

编译安装HAProxy

• 源码下载地址：http://www.haproxy.org/download/2.0/src/haproxy-2.0.26.tar.gz

# HAProxy 1.8及1.9版本编译参数:
make ARCH=x86_64 TARGET=linux2628 USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1 USE_SYSTEMD=1 USE_CPU_AFFINITY=1 PREFIX=/usr/local/haproxy

# HAProxy 2.0以上版本编译参数:
yum install -y gcc openssl-devel pcre-devel systemd-devel

tar xvf haproxy-2.0.26.tar.gz

cd haproxy-2.0.26

# 查看安装方法
ll Makefile

cat README

cat INSTALL

# 编译
make -j 4 TARGET=linux-glibc USE_OPENSSL=1 USE_ZLIB=1 USE_LUA=1 USE_PCRE=1 USE_SYSTEMD=1 LUA_INC=/usr/local/src/lua-5.3.5/src LUA_LIB=/usr/local/src/lua-5.3.5/src

# 安装
make install PREFIX=/apps/haproxy

ln -s /apps/haproxy/sbin/haproxy /usr/sbin/

haproxy -v

• HAProxy启动文件

cat /usr/lib/systemd/system/haproxy.service
[Unit]
Description=HAProxy Load Balancer
After=syslog.target network.target

[Service]
ExecStartPre=/usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -c -q
ExecStart=/usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /var/lib/haproxy/haproxy.pid
ExecReload=/bin/kill -USR2 $MAINPID

[Install]
WantedBy=multi-user.target

mkdir /var/lib/haproxy/

mkdir /etc/haproxy/

# 配置man日志
vim /etc/man_db.conf
MANDATORY_MANPATH     /apps/haproxy/share/man/

# 更新man数据库
mandb

# 配置文件
vim /etc/haproxy/haproxy.cfg
global
  maxconn 100000
  chroot /apps/haproxy
  stats socket /var/lib/haproxy/haproxy.sock mode 600 level admin
  #uid 99
  #gid 99
  user haproxy
  group haproxy
  daemon
  # nbproc 4
  # cpu-map 1 0
  # cpu-map 2 1
  # cpu-map 3 2
  # cpu-map 4 3
  pidfile /var/lib/haproxy/haproxy.pid
  log 127.0.0.1 local2 info

defaults
  option http-keep-alive
  option forwardfor
  maxconn 100000
  mode http
  timeout connect 300000ms
  timeout client 300000ms
  timeout server 300000ms

listen stats
  mode http
  bind 0.0.0.0:9999
  stats enable
  log global
  stats uri    /haproxy-status
  stats auth   haadmin:123456

listen web_port
  bind 10.0.0.7:80
  mode http
  log global
  server web1 127.0.0.1:8080 check inter 3000 fall 2 rise 5


useradd -r -s /sbin/nologin -d /var/lib/haproxy haproxy

systemctl daemon-reload

systemctl enable --now haproxy

Haproxy—缺少/run/haproxy目录，haproxy服务启动失败

问题描述：
搭建haproxy的机器，因出现故障重启了，然后发现haproxy服务出现异常。重新启动haproxy服务的时候出现下面的错误：

[[email protected] log]# service haproxy start
Starting haproxy (via systemctl):  Job for haproxy.service failed because the control process exited with error code. See "systemctl status haproxy.service" and "journalctl -xe" for details.
                                                           [失败]

使用ps -ef查看进程，服务进程没起来：

[[email protected] log]# 
[[email protected] log]# ps -ef|grep haproxy
106      12735 12669  0 09:29 ?        00:00:00 haproxy -p /var/run/haproxy.pid -f /etc/haproxy/haproxy.cfg -sf 26
root     13002 12475  0 09:32 pts/0    00:00:00 grep --color=auto haproxy
[[email protected] log]# 

执行 systemctl status haproxy.service，出现下面的结果：

[[email protected] run]# systemctl status haproxy.service
● haproxy.service - SYSV: HAProxy is a free, very fast and reliable solution offering high availability, load balancing, and proxying for TCP and  HTTP-based applications
   Loaded: loaded (/etc/rc.d/init.d/haproxy; bad; vendor preset: disabled)
   Active: failed (Result: exit-code) since 四 2018-05-10 10:33:24 CST; 9s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 17614 ExecStart=/etc/rc.d/init.d/haproxy start (code=exited, status=1/FAILURE)

5月 10 10:33:24 dev115 systemd[1]: Starting SYSV: HAProxy is a free, very fast and reliable solution offering high availability, load balancing, and proxying for TCP and  HTTP-based applications...
5月 10 10:33:24 dev115** haproxy[17614]: Starting haproxy: [ALERT] 129/103324 (17622) : Starting frontend GLOBAL: cannot bind UNIX socket [/run/haproxy/admin.sock]**
5月 10 10:33:24 dev115 systemd[1]: haproxy.service: control process exited, code=exited status=1
5月 10 10:33:24 dev115 haproxy[17614]: [失败]
5月 10 10:33:24 dev115 systemd[1]: Failed to start SYSV: HAProxy is a free, very fast and reliable solution offering high availability, load balancing, and proxying for TCP and  HTTP-based applications.
5月 10 10:33:24 dev115 systemd[1]: Unit haproxy.service entered failed state.
5月 10 10:33:24 dev115 systemd[1]: haproxy.service failed.
[[email protected] run]# 

根据报错中的“5月 10 10:33:24 dev115 haproxy[17614]: Starting haproxy: [ALERT] 129/103324 (17622) : Starting frontend GLOBAL: cannot bind UNIX socket [/run/haproxy/admin.sock]”这一句，经过链接https://stackoverflow.com/questions/30101075/haproxy-doesnt-start-can-not-bind-unix-socket-run-haproxy-admin-sock 的解答：

Haproxy needs to write to /run/haproxy/admin.sock but it wont create the directory for you. Create the directory /run/haproxy/ first or set stats socket to a different path.

原来是/run下面的haproxy目录丢失了，所以手动在/run目录下面创建了一盒haproxy目录，并且改权限为haproxy：haproxy

[[email protected] run]# mkdir haproxy
[[email protected] run]# chown haproxy:haproxy -P haproxy/

然后重新启动haproxy，成功：

[[email protected] run]# cd /etc/init.d
[[email protected] init.d]# ./haproxy start
Starting haproxy:                                          [  确定  ]
[[email protected] init.d]# 

检查服务进程，进程已经起来了：

[[email protected] haproxy]# ps -ef|grep haproxy
106      17574 17509  0 10:33 ?        00:00:05 haproxy -p /var/run/haproxy.pid -f /etc/haproxy/haproxy.cfg -sf 26
haproxy  17858     1  0 10:36 ?        00:00:00 /usr/local/haproxy/sbin/haproxy -D -f /etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid
root     20764 12475  0 11:20 pts/0    00:00:00 grep --color=auto haproxy
[[email protected] haproxy]#