Keepalived+Haproxy搭建高可用负载均衡
Posted 2206
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了Keepalived+Haproxy搭建高可用负载均衡相关的知识，希望对你有一定的参考价值。
global_defs {
notification_email
{
[email protected]
}
notification_email_from [email protected]
smtp_server smtp.163.com
stmp_connect_timeout 30
router_id lnmp_node2
}
# 检测haproxy脚本
vrrp_script chk_haproxy {
script "/etc/keepalived/check_haproxy.sh"
interval 2
weight 2
}
# 服务
vrrp_instance VIP_1 {
state BACKUP #设置为备用服务器
interface eth0 #监测网络接口
virtual_router_id 51 #主、备必须一样
priority 90 #主、备机取不同的优先级，主机值较大，备份机值较小,值越大优先级越高
advert_int 1 #VRRP Multicast广播周期秒数
authentication {
auth_type PASS #VRRP认证方式，主备必须一致
auth_pass 1111 #密码
}
track_script {
chk_haproxy # 执行监控的服务
}
virtual_ipaddress {
192.168.0.200 #漂移IP地址
}
}
7.3、添加Haproxy检测脚本 vim /etc/keepalived/check_haproxy.sh 添加以下内容
#!/bin/bash if [ $(ps -C haproxy --no-header | wc -l) -eq 0 ]; then /etc/init.d/haproxy start fi sleep 2 if [ $(ps -C haproxy --no-header | wc -l) -eq 0 ]; then /etc/init.d/keepalived stop fi
7.4、给check_haproxy.sh脚本赋值运行权限（MASTER和BACKUP一致）
chmod +x /etc/keepalived/check_haproxy.sh
7.5、允许两台服务器vrrp包通过防火墙，如果关闭防火墙则跳过（两台服务器上都配置）
MASTER vim /etc/sysconfig/iptables -A INPUT -i eth0 -p vrrp -s 192.168.0.111 -j ACCEPT BACKUP vim /etc/sysconfig/iptables -A INPUT -i eth0 -p vrrp -s 192.168.0.110 -j ACCEPT 重启防火墙 service iptables restart
8、启动keepalived服务
service keepalived start
8.1、查看服务器多了一个虚拟IP，keepalived配置成功
MASTER ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:d9:a8:bd brd ff:ff:ff:ff:ff:ff
inet 192.168.0.110/24 brd 192.168.0.255 scope global eth0
inet 192.168.0.200/32 scope global eth0
inet6 fe80::20c:29ff:fed9:a8bd/64 scope link
valid_lft forever preferred_lft forever
BACKUP ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:d9:8f:72 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.109/24 brd 192.168.0.255 scope global eth0
inet6 fe80::20c:29ff:fed9:8f72/64 scope link
valid_lft forever preferred_lft forever
8.2、查看Keepalived日志
tail -f /var/log/messages
9、yum方式安装haproxy
yum install -y haproxy
9.2、查看haproxy版本信息
rpm -qi haproxy 或 haproxy -version Name : haproxy Relocations: (not relocatable) Version : 1.5.4 Vendor: CentOS Release : 3.el6 Build Date: 2016年05月11日 星期三 03时17分37秒 Install Date: 2016年08月24日 星期三 05时34分08秒 Build Host: worker1.bsys.centos.org Group : System Environment/Daemons Source RPM: haproxy-1.5.4-3.el6.src.rpm Size : 2552550 License: GPLv2+ Signature : RSA/SHA1, 2016年05月12日 星期四 18时49分33秒, Key ID 0946fca2c105b9de Packager : CentOS BuildSystem <http://bugs.centos.org> URL : http://www.haproxy.org/ Summary : HAProxy is a TCP/HTTP reverse proxy for high availability environments Description : HAProxy is a TCP/HTTP reverse proxy which is particularly suited for high availability environments. Indeed, it can: - route HTTP requests depending on statically assigned cookies - spread load among several servers while assuring server persistence through the use of HTTP cookies - switch to backup servers in the event a main one fails - accept connections to special ports dedicated to service monitoring - stop accepting connections without breaking existing ones - add, modify, and delete HTTP headers in both directions - block requests matching particular patterns - persists clients to the correct application server depending on application cookies - report detailed status as html pages to authenticated users from a URI intercepted from the application
9.2、查看haproxy位置
rpm -ql haproxy
10、添加独立日志 vim /etc/rsyslog.conf 在底部添加以下配置信息
# haproxy # Provides UDP syslog reception $ModLoad imudp $UDPServerRun 514 # 启动udp，启动端口后将作为服务器工作 # # Provides TCP syslog reception $ModLoad imtcp $InputTCPServerRun 514 # 启动tcp监听端口 local2.* /var/log/haproxy.log
10.1、重启日志服务
service rsyslog restart
10.2、vim haproxy.cfg 在global端中需要添加此行
log 127.0.0.1 local2
11、配置防火墙，允许80，1080端口访问，添加以下两行（测试可以直接关闭防火墙）
-A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 1080 -j ACCEPT
11.1、重启防火墙
service iptables restart
15、编辑配置文件 vim /etc/haproxy/haproxy.cfg
15.1、一个最简单的http服务的配置
global log 127.0.0.1 local2 # 定义日志 chroot /var/lib/haproxy pidfile /var/run/haproxy.pid maxconn 4000 user haproxy group haproxy daemon stats socket /var/lib/haproxy/stats defaults mode http log global option httplog option dontlognull option http-server-close option forwardfor except 127.0.0.0/8 option redispatch retries 3 timeout http-request 10s timeout queue 1m timeout connect 10s timeout client 1m timeout server 1m timeout http-keep-alive 10s timeout check 10s maxconn 3000 frontend webser #webser为名称 option forwardfor bind *:80 default_backend webserver backend webserver balance roundrobin #使拥roundrobin 算法 server app1 192.168.1.120:80 check server app2 192.168.1.121:80 check
15.2、haproxy统计页面的输出机制
frontend webser log 127.0.0.1 local2 option forwardfor bind *:80 default_backend webserver backend webserver cookie node insert nocache balance roundrobin server app1 192.168.0.120:80 check cookie node1 intval 2 rise 1 fall 2 server app2 192.168.0.121:80 check cookie node2 intval 2 rise 1 fall 2 listen statistics bind *:8009 # 自定义监听端口 stats enable # 启用基于程序编译时默认设置的统计报告 stats auth admin:admin # 统计页面用户名和密码设置 stats uri /admin?stats # 自定义统计页面的URL，默认为/haproxy?stats stats hide-version # 隐藏统计页面上HAProxy的版本信息 stats refresh 30s # 统计页面自动刷新时间 stats admin if TRUE #如果认证通过就做管理功能，可以管理后端的服务器 stats realm Hapadmin # 统计页面密码框上提示文本，默认为Haproxy\ Statistics
15.3、静态与动态请求分离
# web服务 frontend webservs # 绑定80端口，域名不限 bind *:80 # 定义静态规则 acl url_static path_beg -i /static /images /javascript /stylesheets acl url_static path_end -i .jpg .gif .png .css .js .html acl host_static hdr_beg(host) -i img. imgs. video. videos. ftp. image. download. # 定义动态规则 acl url_php path_end -i .php # 后端请求归纳 use_backend static if url_static or host_static use_backend dynamic if url_php # 默认动态组 default_backend dynamic # 静态请求处理 backend static # 分配算法（轮流分配） balance roundrobin # 实际处理请求的服务器列表 server node1 192.168.0.120:80 check maxconn 3000 # 动态请求处理 backend dynamic # 分配算法（轮流分配） balance roundrobin # 实际处理请求的服务器列表 server node1 192.168.0.121:80 check maxconn 3000 server node2 192.168.0.122:80 check maxconn 3000
15.4、http完整配置负载均衡
#--------------------------------------------------------------------- # Global settings #--------------------------------------------------------------------- global # to have these messages end up in /var/log/haproxy.log you will # need to: # # 1) configure syslog to accept network log events. This is done # by adding the ‘-r‘ option to the SYSLOGD_OPTIONS in # /etc/sysconfig/syslog # # 2) configure local2 events to go to the /var/log/haproxy.log # file. A line like the following can be added to # /etc/sysconfig/syslog # # local2.* /var/log/haproxy.log # log 127.0.0.1 local2 chroot /var/lib/haproxy pidfile /var/run/haproxy.pid maxconn 4000 user haproxy group haproxy daemon defaults mode http log global option httplog option dontlognull option http-server-close option forwardfor except 127.0.0.0/8 option redispatch retries 3 timeout http-request 10s timeout queue 1m timeout connect 10s timeout client 1m timeout server 1m timeout http-keep-alive 10s timeout check 10s maxconn 30000 listen stats mode http bind 0.0.0.0:1080 stats enable stats hide-version stats uri /haproxyadmin?stats stats realm Haproxy\ Statistics stats auth admin:admin stats admin if TRUE frontend http-in bind *:80 mode http log global option httpclose option logasap #不等待响应结束就记录日志，表示提前记录日志，一般日志会记录响应时长，此不记录响应时长 option dontlognull #不记录空信息 capture request header Host len 20 #记录请求首部的前20个字符 capture request header Referer len 60 #referer跳转引用，就是上一级 default_backend servers frontend healthcheck bind :1099 #定义外部检测机制 mode http option httpclose option forwardfor default_backend servers backend servers balance roundrobin server websrv1 192.168.0.120:80 check maxconn 2000 server websrv2 192.168.0.121:80 check maxconn 2000
15.5、mysql完整配置负载均衡
#--------------------------------------------------------------------- # Global settings #--------------------------------------------------------------------- global # to have these messages end up in /var/log/haproxy.log you will # need to: # # 1) configure syslog to accept network log events. This is done # by adding the ‘-r‘ option to the SYSLOGD_OPTIONS in # /etc/sysconfig/syslog # # 2) configure local2 events to go to the /var/log/haproxy.log # file. A line like the following can be added to # /etc/sysconfig/syslog # # local2.* /var/log/haproxy.log # log 127.0.0.1 local2 chroot /var/lib/haproxy pidfile /var/run/haproxy.pid maxconn 4000 user haproxy group haproxy daemon defaults mode tcp log global option httplog option dontlognull retries 3 timeout http-request 10s timeout queue 1m timeout connect 10s timeout client 1m timeout server 1m timeout http-keep-alive 10s timeout check 10s maxconn 600 listen stats mode http bind 0.0.0.0:1080 stats enable stats hide-version stats uri /haproxyadmin?stats stats realm Haproxy\ Statistics stats auth admin:admin stats admin if TRUE frontend mysql bind *:3306 mode tcp log global default_backend mysqlservers backend mysqlservers balance leastconn server dbsrv1 192.168.1.120:3306 check port 3306 intval 2 rise 1 fall 2 maxconn 300 server dbsrv2 192.168.1.121:3306 check port 3306 intval 2 rise 1 fall 2 maxconn 300
16、启动haproxy服务
service haproxy start
17、查看统计页面
http://192.168.0.200:1080/haproxyadmin?stats 用户名和密码 admin
18、查看Haproxy日志
tail -f /var/log/haproxy.log
19、Haproxy配置信息 MASTR 与 BACKUP配置完全相同
20、访问服务器VIP地址会自动分配到不同服务器进行处理
http://192.168.0.200 1、关闭MASTER服务，BACKUP会自动升级为MASTER接替服务。启动MASTER的Keepalived服务，会自动切回原来的MASTER服务器。 2、关闭Haproxy服务，脚本会尝试启动Haproxy服务，如果启动失败则关闭Keepalived服务，让备用服务器接替。
到这里一个完整的web负载均衡服务器就配置完成了，Haproxy主要做服务分配，Keepalived做双机热备，Keepalived还可以配置成双主热备。在keepalived中检测Haproxy是否可用，不可用是否关闭Keepalived服务器，具体可以根据自己业务做处理。
以上是关于Keepalived+Haproxy搭建高可用负载均衡的主要内容，如果未能解决你的问题，请参考以下文章
搭建高可用负载均衡器: haproxy+keepalived
Kubernetes云原生实战03 搭建高可用负载均衡器(Keepalived 和 HAproxy)
Kubernetes云原生实战03 搭建高可用负载均衡器(Keepalived 和 HAproxy)
Kubernetes云原生实战03 搭建高可用负载均衡器(Keepalived 和 HAproxy)