MariaDB Galera Cluster部署实践

Posted

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了MariaDB Galera Cluster部署实践相关的知识,希望对你有一定的参考价值。

官方文档:

http://galeracluster.com/documentation-webpages/index.html

一、 Galera Cluster的工作原理

主要关注点是数据一致性。 事务既可以应用于每个节点,也可以不全部应用。 所以,只要它们配置正确,数据库保持同步。

Galera复制插件不同于传统的mysql复制,可以解决多个问题,包括多主写入冲突,复制滞后和主从不同步。

技术分享图片

在典型的Galera集群实例中,应用程序可以写入集群中的任何节点,然后通过基于认证的复制将事务提交(RBR事件)应用于所有服务器。

使用组通信和事务排序技术,基于认证的复制是同步数据库复制的另一种方法

二、Galera集群安装

说明:Galera集群至少需要三个节点的服务器硬件。

node-12:10.71.11.12
node-13:10.71.11.13
node-14:10.71.11.14

操作系统:

centos7

内核版本:

 3.10.0-693.21.1.el7.x86_64

开始安装

说明:为集群中的每个节点执行以下步骤。文档以node-12配置为例
1.编辑/etc/hosts文件,配置节点互相解析

[[email protected] ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
10.71.11.12  node-12
10.71.11.13  node-13
10.71.11.14  node-14

2.关闭节点防火墙

[[email protected] ~]# cat /etc/sysconfig/selinux |grep di
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
#     minimum - Modification of targeted policy. Only selected processes are protected.

3.配置Galera集群yum源

[[email protected] ~]# cat /etc/yum.repos.d/galera.repo 
[galera]
name = Galera
baseurl = http://releases.galeracluster.com/galera-3/centos/7/x86_64
gpgkey = http://releases.galeracluster.com/GPG-KEY-galeracluster.com
gpgcheck = 1
[mysql-wsrep]
name = MySQL-wsrep
baseurl =  http://releases.galeracluster.com/mysql-wsrep-5.6/centos/7/x86_64
gpgkey = http://releases.galeracluster.com/GPG-KEY-galeracluster.com
gpgcheck = 1

4.更新yum源缓存

yum makecache

5.安装服务

yum install mysql-wsrep-shared-5.6

说明:如果节点上之前安装过mysql,执行上面命令可能会安装包冲突报错
6.以node-12为例,编辑/etc/my.cnf,添加如下配置

[[email protected] ~]# cat /etc/my.cnf
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
binlog_format=row
bind-address=10.71.11.12
default_storage_engine=innodb
innodb_autoinc_lock_mode=2
innodb_flush_log_at_trx_commit=0
innodb_buffer_pool_size=122M
wsrep_provider=/usr/lib64/galera-3/libgalera_smm.so
wsrep_provider_options="gcache.size=300M; gcache.page_size=300M"
wsrep_sst_method=rsync
wsrep_cluster_name=MyCluster
wsrep_cluster_address="gcomm://10.71.11.12,10.71.11.13,10.71.11.14"
wsrep_node_name=node-12
wsrep_node_address="10.71.11.12"
[mysql_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

说明:其他另外node-13和node-14依据1-6步配置,需要修改/etc/my.cnf与每个节点对应

系统内存配置说明

Galera Cluster的内存要求很难准确预测。 它使用的特定内存量可能会有很大差异,具体取决于给定节点接收的负载。 如果Galera Cluster尝试使用比节点更多的内存,则mysqld实例会崩溃。
保护节点系统崩溃的方法是确保服务有足够的swap 空间分区或者交换文件

检查系统可用交换空间
技术分享图片
技术分享图片
技术分享图片

如果系统swap分区不满足环境需求,可以进行如下操作配置swap分区
说明:集群的中每个节点都要进行相关swap分区设置

1.根据需求在根目录下创建一个2G大小swap空文件

[[email protected] ~]#fallocate -l 2048M /swapfile

或者执行

[[email protected] ~]#dd if=/dev/zero of=/swapfile bs=1M count=2048

设置swap交换分区大小
2.设置swap文件权限

[[email protected] ~]#chown 600 /swapfile
ll /swapfile 
-rw-r--r-- 1 600 root 2147483648 Apr 17 09:13 /swapfile

3.格式化swapfile

[[email protected] ~]# mkswap /swapfile
Setting up swapspace version 1, size = 2097148 KiB
no label, UUID=6341a320-1804-4fe1-8c96-7c22fe270eeb

4 .激活swapfile

[[email protected] ~]# swapon /swapfile
swapon: /swapfile: insecure permissions 0644, 0600 suggested.
swapon: /swapfile: insecure file owner 600, 0 (root) suggested.
  1. 编辑/etc/fstab,在文件底部加入如下配置
    /swapfile none swap defaults 0 0

    技术分享图片
    6.验证swap交换分区配置文件是否成功

    [[email protected] ~]# swapon --summary
    Filename                Type        Size    Used    Priority
    /swapfile                               file    2097148 0   -1

    说明:由于现在部署Galera集群的节点环境部署过其他环境,这里说下删除swap分区的步骤

[[email protected] ~]# swapon --summary
Filename                Type        Size    Used    Priority
/openstack/swap.img                     file    4194300 0   0
/swapfile                               file    2097148 0   -1
[[email protected] ~]# swapoff /openstack/swap.img   ##首先停止swap分区
[[email protected] ~]# rm -rf /openstack/swap.img  ##删除swap分区文件

[[email protected] ~]# swapon --summary
Filename                Type        Size    Used    Priority
/swapfile                               file    2097148 0   -1

技术分享图片

三、Galera集群初始化

启动集群

说明:完成安装和配置Galera Cluster后,需要使用--wsrep-new-cluster选择一个节点上启动mysqld。 这将会初始化集群的核心组件。 在此之后启动的每个节点都将连接到该组件并开始复制
1.在node-12上使用参数--wsrep-new-cluster启动数据库服务

[[email protected] mysql]# service mysql start --wsrep-new-cluster

技术分享图片
数据库初始化报错解决办法,删除/var/lib/mysq/目录下的logfile
技术分享图片
注意警告:只有在初始化主要组件时才使用--wsrep-new-cluster参数。 如果希望节点连接到现有群集时,不要执行这条命令。
2.在node-12节点启动数据库服务后,登录mysql数据库库,通过wsrep_cluster_size参数来判断启动是否成功。

SHOW STATUS LIKE ‘wsrep_cluster_size‘;

技术分享图片
说明:参数wsrep_cluster_size数值代表目前galera集群当前节点在线数

3.向galera集群添加节点,分别在node-12和node-13上执行下面命令启动数据库服务

[[email protected] mysql]# service mysql start
Starting MySQL.Logging to ‘/var/lib/mysql/node-13.err‘.
....... SUCCESS!
[[email protected] yum.repos.d]# service mysql start
Starting MySQL.Logging to ‘/var/lib/mysql/node-14.err‘.
...... SUCCESS!

然后node-13或者node-14上登录mysql数据库,查看wsrep_cluster_size的数值

[[email protected] yum.repos.d]# mysql -uroot -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 4
Server version: 5.6.39 MySQL Community Server (GPL), wsrep_25.22

Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type ‘help;‘ or ‘\h‘ for help. Type ‘\c‘ to clear the current input statement.

mysql> SHOW STATUS LIKE ‘wsrep_cluster_size‘;

技术分享图片
到此,galera集群搭建成功

集群复制测试

说明:要测试Galera群集是否正常工作
1.在数据库客户端上,验证所有节点是否已相互连接

[[email protected] ~]# mysql -uroot
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 4
Server version: 5.6.39 MySQL Community Server (GPL), wsrep_25.22

Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type ‘help;‘ or ‘\h‘ for help. Type ‘\c‘ to clear the current input statement.

mysql> show status like ‘wsrep_%‘;

技术分享图片

技术分享图片

参数说明:
wsrep_local_state_comment参数中Synced表示该节点已经连接到集群并能正常运行
wsrep_cluster_size的数值3表示galera集群中有3个节点都在线
wsrep_ready中on表示此节点连接到集群并能够处理事务。

2.在node-12上创建一个表并插入数据

[[email protected] mysql]# mysql
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 5
Server version: 5.6.39 MySQL Community Server (GPL), wsrep_25.22

Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type ‘help;‘ or ‘\h‘ for help. Type ‘\c‘ to clear the current input statement.

mysql> CREATE DATABASE galeratest;
Query OK, 1 row affected (0.01 sec)

mysql> USE galeratest;
Database changed
mysql> CREATE TABLE test_table (id INT PRIMARY KEY AUTO_INCREMENT,msg TEXT ) ENGINE=InnoDB;
Query OK, 0 rows affected (0.00 sec)

mysql> INSERT INTO test_table (msg) VALUES ("Hello my dear cluster.");
Query OK, 1 row affected (0.00 sec)

mysql> INSERT INTO test_table (msg) VALUES ("Hello, again, cluster dear."); 
Query OK, 1 row affected (0.00 sec)
  1. 在node-13或者ndoe-14查看三个节点能够同步node-12上的数据操作
    
    [[email protected] mysql]# mysql
    Welcome to the MySQL monitor.  Commands end with ; or \g.
    Your MySQL connection id is 4
    Server version: 5.6.39 MySQL Community Server (GPL), wsrep_25.22

Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type ‘help;‘ or ‘\h‘ for help. Type ‘\c‘ to clear the current input statement.

mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| galeratest |
| mysql |
| performance_schema |
| test |
+--------------------+
5 rows in set (0.00 sec)

mysql> use galeratest;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed

mysql> SELECT * FROM test_table;
+----+-----------------------------+
| id | msg |
+----+-----------------------------+
| 1 | Hello my dear cluster. |
| 4 | Hello, again, cluster dear. |
+----+-----------------------------+
2 rows in set (0.00 sec)

![](http://i2.51cto.com/images/blog/201804/24/52ee1a95b756059ec79e0fda5ec78e1a.png?x-oss-process=image/watermark,size_16,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_100,g_se,x_10,y_10,shadow_90,type_ZmFuZ3poZW5naGVpdGk=)

#### 三节点Galera集群故障模拟

1.模拟单个mysqld进程的崩溃,在node-14上kill掉mysqld进程

[[email protected] mysql]# service mysql status
SUCCESS! MySQL running (72075)
[[email protected] mysql]# killall -9 mysqld
-bash: killall: command not found

安装killall命令

[[email protected] mysql]# yum install psmisc -y
[[email protected] mysql]# killall -9 mysqld
[[email protected]]# /usr/bin/mysqld_safe: line 183: 72075 Killed nohup /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --user=mysql --log-error=node-14.err --pid-file=/var/lib/mysql/node-14.pid --socket=/var/lib/mysql/mysql.sock --wsrep_start_position=00000000-0000-0000-0000-000000000000:-1 < /dev/null > /dev/null 2>&1


![](http://i2.51cto.com/images/blog/201804/24/759be41283f0a7de0e34db84917ca364.png?x-oss-process=image/watermark,size_16,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_100,g_se,x_10,y_10,shadow_90,type_ZmFuZ3poZW5naGVpdGk=)
此时到node-12上查看集群状态,显示有2个在线节点
![](http://i2.51cto.com/images/blog/201804/24/c59e5a05427fbaaddbaf71c968e1edc3.png?x-oss-process=image/watermark,size_16,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_100,g_se,x_10,y_10,shadow_90,type_ZmFuZ3poZW5naGVpdGk=)

把故障节点加入galera 集群
![](http://i2.51cto.com/images/blog/201804/24/1f807aaa58d5479be986270f09f7a2dd.png?x-oss-process=image/watermark,size_16,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_100,g_se,x_10,y_10,shadow_90,type_ZmFuZ3poZW5naGVpdGk=)

以上是关于MariaDB Galera Cluster部署实践的主要内容,如果未能解决你的问题,请参考以下文章

MariaDB Galera Cluster部署实践

Mariadb Galera Cluster 部署

MariaDB Galera Cluster 之二 MariaDB Galera Cluster主要功能优缺点和对比

初见 MariaDB Galera Cluster

MariaDB-Galera-cluster实现mariadb高可用的详细步骤

Percona Mysql Galera多读写集群部署生产环境实记