corosync+pacemaker配置高可用集群

Posted 2020-10-26

一、环境介绍：

Node1：

CentOS6      192.168.9.168   192-168-9-168

CentOS6      192.168.9.176   192-168-9-176

VIP： 192.168.9.144

corosync和pacemaker为yum安装

二、环境部署

服务器初始化脚本执行（主要时间同步，防火墙，主机名，ip配置，关闭不必要服务等）

1，更改yum源

wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-6.repo

yum clean all

2，安装epel

rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm

rpm -Uvh http://mirrors.kernel.org/fedora-epel/epel-release-latest-6.noarch.rpm

3，设置主机名

hostname

vim /etc/sysconfig/network

4，绑定host

cat /etc/hosts

192.168.9.168 192-168-9-168

192.168.9.176 192-168-9-176

可配置秘钥通信：

ssh-keygen -P ''

ssh-copy-id -i /root/.ssh/id_rsa.pub [email protected]

ssh 192.168.9.168 'date'

5，安装corosync+pacemaker

yum install -y corosync pacemaker

安装httpd

yum install -y httpd

6，配置

# grep -v "^[:space:]\{0,\}.*#" corosync.conf | grep -v ^$

compatibility: whitetank

totem {

version: 2

secauth: on

threads: 0

interface {

ringnumber: 0

bindnetaddr: 192.168.9.0

mcastaddr: 239.255.1.91

mcastport: 5405

ttl: 1

}

}

logging {

fileline: off

to_stderr: no

to_logfile: yes

logfile: /var/log/cluster/corosync.log

to_syslog: no

debug: off

timestamp: on

logger_subsys {

subsys: AMF

debug: off

}

}

service {

       ver: 0

       name: pacemaker

}

aisexec {

       user: root

       group: root

}

7，启动

service corosync start

8，查看启动是否正常

crm_mon

tail /var/log/cluster/corosync.log

grep -e "Corosync Cluster Engine" -e "configuration file" /var/log/cluster/corosync.log

grep TOTEM /var/log/cluster/corosync.log

grep ERROR: /var/log/cluster/corosync.log | grep -v unpack_resources

grep pcmk_startup /var/log/cluster/corosync.log

9，crm使用方法

http://blog.51yip.com/server/1680.html

（1）yum安装

cd /etc/yum.repos.d/ ; wget http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/network:ha-clustering:Stable.repo

yum install -y crmsh*

（2）×××下载包

http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/noarch/

yum install -y *.rpm

（3）crm使用

# crm     #输入crm命令，进入crm sh 模式

crm(live)# help     #输入help查看帮助

crm(live)# configure     #输入configure进入configure模式下，

crm(live)configure#         #敲两下tab键就会显示configure下全部命令

crm(live)configure# help node     

crm(live)configure# cd ../        #返回上一级

1）查看集群状态

crm(live)# status

2）查看一下CIB配置：

crm(live)# configure

crm(live)configure# show

crm(live)configure# show xml    #查xml格式文件

3）检测一下配置文件是否有错，可以看到刚开始默认STONITH配置错误，因为没有STONITH设备

crm(live)configure# verify

4）查看当前集群系统所支持的资源代理类型

crm(live)# ra

crm(live)ra# classes

5）查看某种类别下的所用资源代理的列表

crm(live)ra# list lsb

crm(live)ra# list ocf heartbeat

crm(live)ra# list ocf pacemaker

6）查看某个资源代理的配置方法，例如查看了heartbeat的ip资源的配置方法：

crm(live)ra# info ocf:heartbeat:IPaddr

7）节点下线，上线

crm node standby

crm node online

crm status

crm node status

8）资源管理

crm(live)# resource

crm(live)# resource

crm(live)resource# status vip

crm(live)resource# migrate vip 192-168-9-176

crm(live)resource# unmigrate

crm(live)resource# stop webservice

9）排列约束

crm(live)configure#  colocation webserver_with_webip inf: webserver webip

10）顺序约束

crm(live)configure# order webip_before_webserver Mandatory: webip webserver

11）位置约束

crm(live)configure# location webip_on_node2 webip 50: 192-168-9-176

10，crmsh配置集群资源

说明：vip，nfs，httpd三个资源配置到资源组，并按顺序启动；双节点投票设置为ignore并添加监控

（1）解决stonith配置错误，设置票数失效操作

crm(live)# configure

crm(live)configure# show

node 192-168-9-168

node 192-168-9-176

property cib-bootstrap-options: \

have-watchdog=false \

dc-version=1.1.15-5.el6-e174ec8 \

cluster-infrastructure="classic openais (with plugin)" \

expected-quorum-votes=2

crm(live)configure# verify

ERROR: error: unpack_resources: Resource start-up disabled since no STONITH resources have been defined

  error: unpack_resources: Either configure some or disable STONITH with the stonith-enabled option

  error: unpack_resources: NOTE: Clusters with shared data need STONITH to ensure data integrity

Errors found during check: config not valid

crm(live)configure# property stonith-enabled=false

crm(live)configure# property no-quorum-policy=ignore

crm(live)configure# verify

crm(live)configure# commit

（2）创建IP地址资源，IP资源是主资源

crm(live)configure# primitive vip ocf:heartbeat:IPaddr params ip=192.168.9.144 nic=eth0 cidr_netmask=24 op monitor interval=10s timeout=20s

crm(live)configure# verify

crm(live)configure# commit

crm(live)configure# show

node 192-168-9-168

node 192-168-9-176

primitive vip IPaddr \

params ip=192.168.9.144 nic=eth0 cidr_netmask=24

property cib-bootstrap-options: \

have-watchdog=false \

dc-version=1.1.15-5.el6-e174ec8 \

cluster-infrastructure="classic openais (with plugin)" \

expected-quorum-votes=2 \

stonith-enabled=false

（3）配置nfs和httpd服务

crm(live)configure# primitive nfs ocf:heartbeat:Filesystem params device=192.168.9.225:/web/hahtml  directory=/var/www/html fstype=nfs op monitor interval=20s timeout=40s

crm(live)configure# primitive httpd lsb:httpd op monitor interval=10s timeout=20s

crm(live)configure# verify

crm(live)configure# commit

crm(live)configure# show

node 192-168-9-168

node 192-168-9-176

primitive httpd lsb:httpd

primitive nfs Filesystem \

params device="192.168.9.225:/web/hahtml" directory="/var/www/html" fstype=nfs

primitive vip IPaddr \

params ip=192.168.9.144 nic=eth0 cidr_netmask=24

property cib-bootstrap-options: \

have-watchdog=false \

dc-version=1.1.15-5.el6-e174ec8 \

cluster-infrastructure="classic openais (with plugin)" \

expected-quorum-votes=2 \

stonith-enabled=false

crm(live)configure# cd

crm(live)# status

Stack: classic openais (with plugin)

Current DC: 192-168-9-168 (version 1.1.15-5.el6-e174ec8) - partition with quorum

Last updated: Fri Mar  2 14:53:37 2018 Last change: Fri Mar  2 14:52:46 2018 by root via cibadmin on 192-168-9-168

, 2 expected votes

2 nodes and 3 resources configured

Online: [ 192-168-9-168 192-168-9-176 ]

Full list of resources:

vip (ocf::heartbeat:IPaddr): Started 192-168-9-168

nfs (ocf::heartbeat:Filesystem): Started 192-168-9-176

httpd (lsb:httpd): Started 192-168-9-168

（4）创建资源组，将三个资源在同一节点启动

crm(live)# configure

crm(live)configure#  group group_webservice vip nfs httpd

crm(live)configure# verify

crm(live)configure# commit

crm(live)configure# show

node 192-168-9-168

node 192-168-9-176

primitive httpd lsb:httpd

primitive nfs Filesystem \

params device="192.168.9.225:/web/hahtml" directory="/var/www/html" fstype=nfs

primitive vip IPaddr \

params ip=192.168.9.144 nic=eth0 cidr_netmask=24

group group_webservice vip nfs httpd

property cib-bootstrap-options: \

have-watchdog=false \

dc-version=1.1.15-5.el6-e174ec8 \

cluster-infrastructure="classic openais (with plugin)" \

expected-quorum-votes=2 \

stonith-enabled=false

crm(live)configure# cd

crm(live)# status

Stack: classic openais (with plugin)

Current DC: 192-168-9-168 (version 1.1.15-5.el6-e174ec8) - partition with quorum

Last updated: Fri Mar  2 14:55:12 2018 Last change: Fri Mar  2 14:55:04 2018 by root via cibadmin on 192-168-9-168

, 2 expected votes

2 nodes and 3 resources configured

Online: [ 192-168-9-168 192-168-9-176 ]

Full list of resources:

Resource Group: group_webservice

    vip (ocf::heartbeat:IPaddr): Started 192-168-9-168

    nfs (ocf::heartbeat:Filesystem): Started 192-168-9-168

    httpd (lsb:httpd): Started 192-168-9-168

11，测试

（1）使节点为standy  测试切换

（2）service corosync stop关闭corosync  测试选票系统

（3）killall httpd服务测试监控