Heartbeat+Drbd+Mysql高可用（HA）集群架构的部署
Posted
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了Heartbeat+Drbd+Mysql高可用（HA）集群架构的部署相关的知识，希望对你有一定的参考价值。
主机环境 redhat6.5 64位
实验环境 服务端1 ip 172.25.25.111 主机名：server1.example.com
服务端2 ip172.25.25.112 主机名：server2.example.com
安装包 heartbeat-3.0.4-2.el6.x86_64.rpm
heartbeat-devel-3.0.4-2.el6.x86_64.rpm
ldirectord-3.9.5-3.1.x86_64.rpm
heartbeat-libs-3.0.4-2.el6.x86_64.rpm
drbd-8.4.3.tar.gz
防火墙状态：关闭
前面的博文中提过mysql的源码安装和drbd的内核模块的添加，有些报错的解决、安装、测试这里不再重复，只是稍微提一下，感兴趣的可以看前面的博文。
1.安装heartbeat、修改主配置文件及测试
1.安装heartbeat、修改主配置文件（服务端1）
[[email protected] mnt]# ls #安装包
heartbeat-3.0.4-2.el6.x86_64.rpm heartbeat-devel-3.0.4-2.el6.x86_64.rpm ldirectord-3.9.5-3.1.x86_64.rpm
heartbeat-libs-3.0.4-2.el6.x86_64.rpm
[[email protected] mnt]# yum install * -y #安装/mnt所有
[[email protected] mnt]# cd /usr/share/doc/heartbeat-3.0.4/ #切换
[[email protected] heartbeat-3.0.4]# ls
apphbd.cf authkeys AUTHORS ChangeLog COPYING COPYING.LGPL ha.cf haresources README
[[email protected] heartbeat-3.0.4]# cp ha.cf haresources authkeys/etc/ha.d/ #拷贝主配置文件、资源配置文件、认证文件到指定目录
[[email protected] heartbeat-3.0.4]# cd /etc/ha.d/
[[email protected] ha.d]# ls
authkeys ha.cf harc haresources rc.d README.config resource.d shellfuncs
[[email protected] ha.d]# vim ha.cf #进入主配置文件
34 logfacility local0
48 keepalive 2
56 deadtime 30
61 warntime 10
71 initdead 60
76 udpport 719 #udp协议端口号
91 bcast eth0
157 auto_failback on
#结点主机名，那个写在前面那个就是主，另一个就是备，如下：服务端1是主，服务端2是备
211 node server1.example.com
212 node server2.example.com
220 ping 172.25.25.250
253 respawn hacluster /usr/lib64/heartbeat/ipfail
259 apiauth ipfail gid=haclient uid=hacluster
[[email protected] ha.d]# vim haresources
150 server1.example.com IPaddr::172.25.25.100/24/eth0 httpd #主机名是主结点的主机名、添加虚拟ip、httpd服务
[[email protected] ha.d]# vim authkeys #进入认证文件
23 auth 1 #认证
24 1 crc #认证方式crc
[email protected] ha.d]# chmod 600 authkeys #修改认证文件的权限
[[[email protected] ha.d]# /etc/init.d/heartbeat start #开启heartbeat
服务端2和服务端1的配置完全相同，可以在服务端2安装好heartbeat的情况下，将服务端1配置好的文件考过去，放在相同的目录下，然后启动服务端2的heartbeat服务，如下;
[[email protected] ha.d]# scp ha.cf haresources authkeys [email protected]:/etc/ha.d/ #拷贝
[email protected]‘s password:
[[email protected] ha.d]# /etc/init.d/heartbeat start #启动（服务端2）
在测试之前，服务端1和服务端2要安装httpd且有内容不同的测试页。注意：不要手动启动httpd，系统会自动启动。
2.测试 172.25.25.100
[[email protected] ha.d]# /etc/init.d/httpd stop #将服务端1的httpd停止，测试
Stopping httpd: [ OK ]
[[email protected] ha.d]# /etc/init.d/httpd start #开启httpd
Starting httpd: [ OK ]
2.编译、安装drbd内核模块及测试
1.编译、安装drbd内核模块
[[email protected] mnt]# ls
drbd-8.4.3.tar.gz
[[email protected] mnt]# tar zxf drbd-8.4.3.tar.gz #解压
[[email protected] mnt]# ls
drbd-8.4.3 drbd-8.4.3.tar.gz
[[email protected] mnt]# cd drbd-8.4.3
[[email protected] drbd-8.4.3]# ./configure --enable-spec --with-km
[[email protected] drbd-8.4.3]# rpmbuild -bb drbd.spec
error: File /root/rpmbuild/SOURCES/drbd-8.4.3.tar.gz: No suchfile or directory
[[email protected] drbd-8.4.3]# cp /mnt/drbd-8.4.3.tar.gz/root/rpmbuild/SOURCES/
[[email protected] drbd-8.4.3]# rpmbuild -bb drbd.spec
[[email protected] drbd-8.4.3]# rpmbuild -bb drbd-km.spec
[[email protected] drbd-8.4.3]# cd /root/rpmbuild/RPMS/x86_64/ #切换目录
[[email protected] x86_64]# ls #生成10个安装包
drbd-8.4.3-2.el6.x86_64.rpm
drbd-bash-completion-8.4.3-2.el6.x86_64.rpm
drbd-debuginfo-8.4.3-2.el6.x86_64.rpm
drbd-heartbeat-8.4.3-2.el6.x86_64.rpm
drbd-km-2.6.32_431.el6.x86_64-8.4.3-2.el6.x86_64.rpm
drbd-km-debuginfo-8.4.3-2.el6.x86_64.rpm
drbd-pacemaker-8.4.3-2.el6.x86_64.rpm
drbd-udev-8.4.3-2.el6.x86_64.rpm
drbd-utils-8.4.3-2.el6.x86_64.rpm
drbd-xen-8.4.3-2.el6.x86_64.rpm
[[email protected] x86_64]# yum install * -y #安装所有的包
[[email protected] x86_64]# scp * [email protected]:/mnt #将包传给服务端2
[[email protected] mnt]# rpm -vih * #安装
Preparing... ########################################### [100%]
1:drbd-utils ########################################### [ 10%]
2:drbd-bash-completion ########################################### [ 20%]
3:drbd-heartbeat ########################################### [ 30%]
4:drbd-pacemaker ########################################### [ 40%]
5:drbd-udev ########################################### [ 50%]
6:drbd-xen ########################################### [ 60%]
7:drbd ###########################################[ 70%]
8:drbd-km-2.6.32_431.el6.########################################### [80%]
9:drbd-km-debuginfo ########################################### [ 90%]
10:drbd-debuginfo ###########################################[100%]
在服务端1和服务端2虚拟机中各添加一块虚拟磁盘或划分一个新的分区，然后进行下面的步骤：
[[email protected] x86_64]# cd /etc/drbd.d/ #切换到drbd的主配置文件目录
[[email protected] drbd.d]# vim dbdata.res #创建一个文件，后缀必须是.res
1 resource dbdata{
2 meta-disk internal;
3 device /dev/drbd1;
4 syncer {
5 verify-alg sha1; #设置主备机之间通信使用的信息算法
6 }
#每个主机的说明以"on"开头,后面是主机名.在后面的{}中为这个主机的配置
7 on server1.example.com{ #主机名
8 disk /dev/vdb; #磁盘名称
9 address172.25.29.1:7789; #ip加端口，端口是指定的
10 }
11 onserver2.example.com{
12 disk /dev/vdb;
13 address172.25.29.2:7789;
14 }
15 }
[[email protected] drbd.d]# scp [email protected]:/etc/drbd.d/
[email protected]‘s password:
#服务端1和服务端2同时初始化 ,同时开启drbd
[[email protected] drbd.d]# drbdadm create-md dbdata #初始化
--== Thank you for participating in the globalusage survey ==--
The server‘s response is:
Writing meta data...
initializing activity log
NOT initializing bitmap
New drbd meta data block successfully created.
[[email protected] drbd.d]# drbdadm create-md dbdata
--== Thank you for participating in the globalusage survey ==--
The server‘s response is:
Writing meta data...
initializing activity log
NOT initializing bitmap
New drbd meta data block successfully created.
[[email protected] drbd.d]# /etc/init.d/drbd start #开启drbd
[[email protected] drbd.d]# /etc/init.d/drbd start
[[email protected] drbd.d]# cat /proc/drbd #查看
version: 8.4.3 (api:1/proto:86-101)
GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build [email protected], 2016-10-08 15:10:25
1: cs:Connectedro:Secondary/Secondary ds:Inconsistent/Inconsistent C r-----
ns:0 nr:0 dw:0 dr:0al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:2097052
[[email protected] drbd.d]# cat /proc/drbd
version: 8.4.3 (api:1/proto:86-101)
GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build [email protected], 2016-10-08 15:10:25
1: cs:Connectedro:Secondary/Secondary ds:Inconsistent/Inconsistent C r-----
ns:0 nr:0 dw:0 dr:0al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:2097052
#从上面可以看出两个都是secondary
[[email protected] drbd.d]# drbdadm primary dbdata --force #强制使服务端1变成primary
[[email protected] drbd.d]# cat /proc/drbd #查看，服务端1和服务端2正在同步
version: 8.4.3 (api:1/proto:86-101)
GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build [email protected], 2016-10-08 15:10:25
1: cs:SyncSource ro:Primary/Secondaryds:UpToDate/Inconsistent C r-----
ns:496640 nr:0 dw:0dr:497304 al:0 bm:30 lo:0 pe:4 ua:0 ap:0 ep:1 wo:f oos:1604508
[===>................]sync‘ed: 23.7% (1604508/2097052)K
finish: 0:00:16 speed:98,508 (98,508) K/sec
[[email protected] drbd.d]# cat /proc/drbd
version: 8.4.3 (api:1/proto:86-101)
GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build [email protected], 2016-10-08 15:10:25
1: cs:SyncSourcero:Primary/Secondary ds:UpToDate/Inconsistent C r---n-
ns:1566436 nr:0 dw:0dr:1569432 al:0 bm:94 lo:0 pe:16 ua:3 ap:0 ep:1 wo:f oos:546716
[=============>......]sync‘ed: 74.1% (546716/2097052)K
finish: 0:00:08 speed:64,596 (64,596) K/sec
[[email protected] drbd.d]# cat /proc/drbd #同步完成，服务端1为primary
version: 8.4.3 (api:1/proto:86-101)
GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build [email protected], 2016-10-08 15:10:25
1: cs:Connectedro:Primary/Secondary ds:UpToDate/UpToDate C r-----
ns:2097052 nr:0 dw:0dr:2097716 al:0 bm:128 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
[[email protected] drbd.d]# cat /proc/drbd
version: 8.4.3 (api:1/proto:86-101)
GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build [email protected], 2016-10-08 15:10:25
1: cs:Connectedro:Secondary/Primary ds:UpToDate/UpToDate C r-----
ns:0 nr:2097052dw:2097052 dr:0 al:0 bm:128 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
[[email protected] drbd.d]# mkfs.ext4 /dev/drbd1 #格式化
3.将drdb、mysql写进heartbeat的文件里且测试
1.将drdb、mysql写进heartbeat的文件（服务端1）
[[email protected] ～]# yum install mysql-server -y #服务端1和服务端2都要安装mysql，不要手动启动
[[email protected] ~]# cat /proc/drbd #服务端1和服务端2要同为secondary
version: 8.4.3 (api:1/proto:86-101)
GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build [email protected], 2016-10-02 15:19:31
1: cs:Connectedro:Secondary/Secondary ds:UpToDate/UpToDate C r-----
ns:171868 nr:0 dw:0dr:171868 al:0 bm:18 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
[[email protected] ~]# cd /etc/ha.d/ #切换到heartbeat的配置文件路径
[[email protected] ha.d]# vim haresources #进入
150 server1.example.com IPaddr::172.25.25.100/24/eth0drbddisk::dbdata Filesystem::/dev/drbd1::/var/lib/mysql::ext4 mysqld #将/dev/drbd1挂载到/var/lib/mysql目录下，开启mysql
[[email protected] ~]# /etc/init.d/heartbeat stop #停止heartbeat
Stopping High-Availability services: Done.
[[email protected] ~]# /etc/init.d/heartbeat status #查看状态
heartbeat is stopped. No process
[[email protected] ~]# /etc/init.d/heartbeat start
Starting High-Availability services: INFO: Resource is stopped
Done.
[[email protected] ~]# /etc/init.d/heartbeat start #开启heartbeat
Starting High-Availability services: INFO: Resource is stopped
Done.
[[email protected] ha.d]# scp [email protected]:/etc/ha.d/
[email protected]‘s password:
服务端2
[[email protected] ha.d]# /etc/init.d/heartbeat stop
Stopping High-Availability services: Done.
[[email protected] ha.d]# /etc/init.d/heartbeat start
Starting High-Availability services: INFO: Resource is stopped
Done.
2.测试
[[email protected] ~]# ping 172.25.25.100 #ping通了
64 bytes from 172.25.29.100: icmp_seq=1 ttl=64 time=0.485 ms
64 bytes from 172.25.29.100: icmp_seq=2 ttl=64 time=0.292 ms
64 bytes from 172.25.29.100: icmp_seq=3 ttl=64 time=0.289 ms
[[email protected] ~]# arp -an|grep 172.25.25.100 #查看ip
? (172.25.25.100) at 52:54:00:ec:8b:36 [ether] on eth0
[[email protected] ~]# ip addr show eth0 #和服务端1的一样
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdiscpfifo_fast state UP qlen 1000
link/ether52:54:00:ec:8b:36 brd ff:ff:ff:ff:ff:ff
inet 172.25.25.111/24brd 172.25.25.255 scope global eth0
inet 172.25.25.100/24brd 172.25.25.255 scope global secondary eth0
inet6fe80::5054:ff:feec:8b36/64 scope link
valid_lft foreverpreferred_lft forever
[[email protected] ~]# df #磁盘也挂载上了
Filesystem 1K-blocks Used Available Use%Mounted on
/dev/mapper/VolGroup-lv_root 7853764 1290388 6164428 18% /
tmpfs 510200 0 510200 0% /dev/shm
/dev/vda1 495844 33467 436777 8% /boot
/dev/drbd1 4128284 95176 3823404 3% /var/lib/mysql
[[email protected] ~]# mysql #mysql也打开了
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| test |
+--------------------+
3 rows in set (0.00 sec)
[[email protected] ha.d]# mysql
ERROR 2002 (HY000): Can‘t connect to local MySQL server throughsocket ‘/var/lib/mysql/mysql.sock‘ (2)
[[email protected] ~]# /etc/init.d/heartbeat stop #当服务端1的heartbeat停止之后
Stopping High-Availability services: Done.
[[email protected] ~]# mysql
ERROR 2002 (HY000): Can‘t connect to local MySQL server throughsocket ‘/var/lib/mysql/mysql.sock‘ (2)
[[email protected] ~]# arp -an|grep 172.25.25.100 #查看ip
? (172.25.25.100) at 52:54:00:85:1a:3b [ether] on eth0
[[email protected] ha.d]# ip addr show eth0 #服务端2在服务
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdiscpfifo_fast state UP qlen 1000
link/ether52:54:00:85:1a:3b brd ff:ff:ff:ff:ff:ff
inet 172.25.25.112/24brd 172.25.25.255 scope global eth0
inet6fe80::5054:ff:fe85:1a3b/64 scope link
valid_lft foreverpreferred_lft forever
[[email protected] ha.d]# df #挂载上了
Filesystem 1K-blocks Used Available Use%Mounted on
/dev/mapper/VolGroup-lv_root 7853764 1188980 6265836 16% /
tmpfs 510200 0 510200 0% /dev/shm
/dev/vda1 495844 33467 436777 8% /boot
/dev/drbd1 4128284 95176 3823404 3% /var/lib/mysql
[[email protected] ~]# /etc/init.d/heartbeat start #当服务端1的heartbeat启动之后
Starting High-Availability services: INFO: Resource is stopped
Done.
[[email protected] ~]# arp -an|grep 172.25.25.100 #又回到了服务端1
? (172.25.25.100) at 52:54:00:ec:8b:36 [ether] on eth0
[[email protected] ~]# ip addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdiscpfifo_fast state UP qlen 1000
link/ether52:54:00:ec:8b:36 brd ff:ff:ff:ff:ff:ff
inet 172.25.25.111/24brd 172.25.25.255 scope global eth0
inet 172.25.25.100/24brd 172.25.25.255 scope global secondary eth0
inet6 fe80::5054:ff:feec:8b36/64scope link
valid_lft foreverpreferred_lft forever
本文出自 “不忘初心，方得始终” 博客，请务必保留此出处http://12087746.blog.51cto.com/12077746/1860294
以上是关于Heartbeat+Drbd+Mysql高可用（HA）集群架构的部署的主要内容，如果未能解决你的问题，请参考以下文章
15 Heartbeat+DRBD+MySQL高可用架构方案与实施过程细节