keepalived_lvs-dr实现discuz负载均衡和高可用
Posted
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了keepalived_lvs-dr实现discuz负载均衡和高可用相关的知识，希望对你有一定的参考价值。
前言：
前一篇文章中由于前端调度器由于只有一台，如果损坏了整个系统将无法对外提供服务，因此它成为了系统的SPOF，需要对其做高可用，此时暂不考虑后端存储故障的情况。经过两个小时的配置，终于将此配置出来，下面将配置流程写下来
由于我的电脑使用的是虚拟机，所以在配置DR模型的时候，需要使用两块网卡，一块网卡用于桥接外网，另外一块使用NAT模式进行实验测试！！！(此处无需关注链接外网的网卡，本地测试NAT模式很实用)
高可用模型：
主备模型：即一台主机位MASTER，另一台主机位BACKUP,只有当MASTER主机宕机时，BACKUP主机取而代之成为MASTER才会开始工作，因此在单主模型中，始终有一台主机不工作，会造成资源浪费。
双主模型：两个单主模型叠加，两台主机互为主备，在前段DNS解析时加上双A记录，即可实现两台主机同时工作，而且实现了高可用，且兼备负载均衡。
此处我们使用主备模型，两模型之间切换过于简单，此处不描述主主模型博客！
拓扑图：
各个主机的IP地址
Host VS1
eno16777736 10.0.0.201/8
gateway: 10.0.0.254
Host VS2
eno16777736 10.0.0.203/8
gateway: 10.0.0.254
Host RS1
eno16777736 10.0.0.101/8 (RIP1)
lo:0 10.0.0.111/32 (VIP)
gateway: 10.0.0.254
Host RS2
eno16777736 10.0.0.102/8 (RIP2)
lo:0 10.0.0.111/32 (VIP)
gateway: 10.0.0.254
Host DB
eno16777736 10.0.0.202/8
gateway: 10.0.0.254
时间同步：
# ntpdate cn.pool.ntp.org # hwclock --sysohc
安装软件：
HostDB
安装二进制mariadb-5.5.46
详细配置请入传送门：http://wscto.blog.51cto.com/11249394/1783131
安装NFS
# yum install -y nfs-utils
HostRS1
安装nginx，注意nginx属于epel源
# yum install-y nginx php-fpm php-mbstring php-mysql nfs-utils mariadb
HostRS2
安装nginx，注意nginx属于epel源
# yum install-y nginx php-fpm php-mbstring php-mysql nfs-utils mariadb
Host VS1
安装keepalived是为了实现高可用，ipvsadm用来查看状态 http用来提供sorry server
# yum install -y keepalived ipvsadm httpd
Host VS2
安装keepalived是为了实现高可用，ipvsadm用来查看状态 http用来提供sorry server
# yum install -y keepalived ipvsadm httpd
HostDB
完成安全初始化后，创建discuz数据库和discuz用户，并授权其可以远程操作数据库
# mysql_secure_installation # mysql -u root -p Enter password: Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 3 Server version: 5.5.46-MariaDB-log MariaDB Server Copyright (c) 2000, 2015, Oracle, MariaDB Corporation Ab and others. Type ‘help;‘ or ‘\h‘ for help. Type ‘\c‘ to clear the current input statement. MariaDB [(none)]> create database discuz; Query OK, 1 row affected (0.00 sec) MariaDB [(none)]> create user ‘discuz‘@‘localhost‘ identified by ‘magedu‘; Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> grant all privileges on discuz.* to ‘discuz‘@‘%‘ identified by ‘magedu‘; Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> flush privileges; Query OK, 0 rows affected (0.00 sec)
配置NFS
# mkdir /nfshare/ # ls -dl /nfshare/ drwxr-xr-x 2 root root 6 May 9 17:01 /nfshare/ # echo "/nfshare/ 10.0.0.101(rw,no_root_squash,async) 10.0.0.102(rw,no_root_squash,async)" > /etc/exports # systemctl start rpcbind <--如果是CentOS-6要使用/etc/init.d/rpcbind start Starting rpcbind: [ OK ] [[email protected] ~]# /etc/init.d/nfs start Starting NFS services: [ OK ] Starting NFS quotas: [ OK ] Starting NFS mountd: [ OK ] Starting NFS daemon: [ OK ] Starting RPC idmapd: [ OK ] # systemctl enable rpcbind <--如果是6 chkconfig rpcbind on # systemctl enable rpcbind <--如果是6 chkconfig nfs on [[email protected] ~]# chkconfig rpcbind --list rpcbind 0:off 1:off 2:on 3:on 4:on 5:on 6:off [[email protected] ~]# chkconfig nfs --list nfs 0:off 1:off 2:on 3:on 4:on 5:on 6:off [[email protected] ~]# showmount -e 127.0.0.1 Export list for 127.0.0.1: /nfshare 172.18.71.102,172.18.71.101 此步骤如果出现问题，请查看《lvs-dr负载均衡Discuz》
解压discuz的程序包至/nfsshare/目录
# mkdir /nfshare/discuz # unzip /Discuz_X3.2_SC_UTF8.zip -d /nfshare/discuz/ # ls /nfshare/discuz/ readme upload utility
Host RS1
我们如果要为RS配置VIP，为了不产生IP冲突，我们要先禁止RS响应VIP的ARP请求
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
在本地回环接口别名上配置VIP,并增添路由
# ip addr add 10.0.0.111/32 broadcast 10.0.0.111 dev lo labal lo:0 # route add -host 10.0.0.111 lo:0
注意:CentOS-7最小安装，默认没有route命令，需要安装net-tools(不用google了，遇到这个问题的都特么是外国人。没错，很坑爹)
测试链接HostDB上的mariadb
# mysql -h 10.0.0.202 -u discuz -p
挂载HostDB上的NFS共享存储目录，属主属组改为apache
# showmount -e 10.0.0.202 Export list for 10.0.0.202: /nfshare 10.0.0.102,10.0.0.101 [[email protected] ~]# mkdir /htdocs [[email protected] ~]# ls -ld /htdocs/ drwxr-xr-x 2 root root 6 May 9 17:05 /htdocs/ [[email protected] ~]# mount -t nfs 10.0.0.202:/nfshare /htdocs [[email protected] ~]# ls /htdocs/ discuz [[email protected] ~]# chown -R apache:apache /htdocs/discuz/
启动php-fpm
# systemctl start php-fpm.service
修改nginx的配置文件
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
events {
worker_connections 1024;
}
http {
log_format main ‘$remote_addr - $remote_user [$time_local] "$request" ‘
‘$status $body_bytes_sent "$http_referer" ‘
‘"$http_user_agent" "$http_x_forwarded_for"‘;
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
include /etc/nginx/conf.d/*.conf;
server {
listen 80;
server_name www.rs1.com;
# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;
location / {
root /htdocs/discuz/upload;
index index.html index.htm index.php;
}
location ~ \.php$ {
root /htdocs/discuz/upload;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
include /etc/nginx/fastcgi.conf;
}
error_page 404 /404.html;
location = /40x.html {
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
}
}
}
# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
# nginx
客户端浏览器访问http://10.0.0.101/install/,在这里就将discuz安装完成Host RS2
与Host RS1的配置相同，此处不再复制
Host VS1
修改keepalived配置文件
! Configuration File for keepalived
global_defs {
notification_email {
[email protected]
}
notification_email_from [email protected]
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
vrrp_mcast_group4 224.0.71.18
}
vrrp_instance VI_1 {
state MASTER
interface eno16777736
virtual_router_id 52
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.111/32 dev eno16777736 label eno16777736:0
}
}
virtual_server 10.0.0.111 80 {
delay_loop 6
lb_algo rr
lb_kind DR
protocol TCP
sorry_server 127.0.0.1 80
real_server 10.0.0.101 80 {
weight 1
HTTP_GET {
url {
path /install/
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 10.0.0.102 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
启动哈个服务Host VS2
修改keepalived配置文件
! Configuration File for keepalived
global_defs {
notification_email {
[email protected]
}
notification_email_from [email protected]
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
vrrp_mcast_group4 224.0.71.18
}
vrrp_instance VI_1 {
state BACKUP
interface eno16777736
virtual_router_id 52
priority 98
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.111/32 dev eno16777736 label eno16777736:0
}
}
virtual_server 10.0.0.111 80 {
delay_loop 6
lb_algo rr
lb_kind DR
protocol TCP
sorry_server 127.0.0.1 80
real_server 10.0.0.101 80 {
weight 2
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 10.0.0.102 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}至此，地址已经可以飘起来了，如果遇到什么麻烦请联系我QQ:2137626
为HostVS1 和HostVS2 提供Sorry页面，并启动服务
# echo "服务维护中,请稍后访问." > /var/www/html/index.html [[email protected] ~]#sed -i "[email protected]^#ServerName.*@ServerName [email protected]" /etc/httpd/conf/httpd.conf [[email protected] ~]# /etc/init.d/httpd start Starting httpd: [ OK ]
测试验证
通过客户端浏览器强制刷新10.0.0.111在HostVS1和2上都可以通过ipvsadm查看，两个后端RS主机轮询的很开心
ipvsadm IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.0.0.111:http rr -> 10.0.0.101:http Route 1 3 6 -> 10.0.0.102:http Route 1 3 0
以上是关于keepalived_lvs-dr实现discuz负载均衡和高可用的主要内容，如果未能解决你的问题，请参考以下文章