Gluster分布式存储--使用

Posted 2020-09-28 2009_boy

实现功能：

把w1和w2上的/ application/nginx-1.10.2/html/blog/wp-content/uploads（wordpress图片目录）目录挂载到n1和n2的/data目录下

1.1.1 准备阶段

IP规划

w1　　172.16.1.121

w2　　172.16.1.122

n1　　172.16.1.123

n2　　172.16.1.124

查看阻碍

[[email protected] ~]# getenforce

Disabled

[[email protected] ~]# /etc/init.d/iptables status

iptables: Firewall is not running.

 

所有服务器都要解析,查看hosts解析情况

[[email protected] ~]# cat /etc/hosts

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4

::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

172.16.1.121 w1

172.16.1.122 w2

172.16.1.123 n1

172.16.1.124 n2

 

yum下载服务(需要特殊的gluster源)

wget -P /etc/yum.repos.d/  http://download.gluster.org/pub/gluster/glusterfs/3.7/LATEST/RHEL/glusterfs-epel.repo

yum install glusterfs-server -y

这里就统一都下载服务端用yum安装即可

一般是有服务端和客户端区分的，想要下载建议去官网下载https://www.gluster.org/ 需要编译安装

1.2 安装部署

开启服务（以下内容在n1、n2上操作即可，客户端不用操作/启动）

[[email protected] ~]# /etc/init.d/glusterd start

Starting glusterd:                                         [  OK  ]

把n2加入池（只在n1上操作）

[[email protected] ~]# gluster peer probe n2

peer probe: success. Host 10.0.0.61 port 24007 already in peer list

查看池子

[[email protected] ~]# gluster peer status

Number of Peers: 1                                                            #显示除自己外的池子里还有几台服务器

 

Hostname: n2                                                                     #服务器的名字

Uuid: e530336c-d1f0-46b6-b431-d932679764d9       #如果磁盘或服务器挂了可以使用UUID恢复数据

State: Peer in Cluster (Connected)

最好用新的磁盘来做卷

插入一块磁盘sdb并创建分区

[[email protected] ~]# fdisk /dev/sdb

Command (m for help): n

Command action

   e   extended

   p   primary partition (1-4)

p

Partition number (1-4): 1

First cylinder (1-130, default 1):

Using default value 1

Last cylinder, +cylinders or +size{K,M,G} (1-130, default 130):

Using default value 130

 

Command (m for help): wq

The partition table has been altered!

 

Calling ioctl() to re-read partition table.

Syncing disks.

创建完了磁盘分区然后格式化

[[email protected] ~]# mkfs.ext4 /dev/sdb1

挂载到要创建卷的目录/data上

[[email protected] ~]# mount -t ext4 /dev/sdb1 /data/

[[email protected] ~]# df -h

Filesystem      Size  Used Avail Use% Mounted on

/dev/sda3       6.9G  3.2G  3.4G  49% /

tmpfs           491M     0  491M   0% /dev/shm

/dev/sda1       190M   33M  147M  19% /boot

/dev/sdb1       988M  1.3M  936M   1% /data

1.2.1 创建复制卷（以下内容只在n1上操作即可，n2不用操作）

[[email protected] ~]# gluster volume create pick replica 2 n1:/data/ n2:/data/ force

volume create: pick: success: please start the volume to access data

 

查看卷的信息

[[email protected] ~]# gluster volume info

Volume Name: pick                                                                          #卷的名字

Type: Replicate                                                                                #卷的类别（属于什么卷）

Volume ID: b4623968-3268-4bb9-b631-409355ebfe10                 #卷的ID

Status: Created                                                                                #卷的状态

Number of Bricks: 1 x 2 = 2                                                            #一共有几个brick

Transport-type: tcp                                                                           #什么协议（TCP/UDP/TCP+UDP）

Bricks:

Brick1: n1:/data                                                                               #第一个brick目录

Brick2: n2:/data                                                                               #第二个brick目录

Options Reconfigured:

performance.readdir-ahead: on

状态Status: Created是关闭的，所以要开启卷

[[email protected] ~]# gluster volume start pick

volume start: pick: success

 

检查是否开启

[[email protected] ~]# gluster volume info

 

Volume Name: pick

Type: Replicate

Volume ID: b4623968-3268-4bb9-b631-409355ebfe10

Status: Started

Number of Bricks: 1 x 2 = 2

Transport-type: tcp

Bricks:

Brick1: n1:/data

Brick2: n2:/data

Options Reconfigured:

performance.readdir-ahead: on

1.2.2 挂载卷

在w1和w2上，把wordpress图片目录挂载上即可（不用开启glusterd）

[[email protected] ~]# mount -t glusterfs n1:/pick /application/nginx/html/blog/wp-content/uploads/

[[email protected] ~]# df -h

Filesystem      Size  Used Avail Use% Mounted on

/dev/sda3       6.9G  3.7G  2.8G  58% /

tmpfs           491M     0  491M   0% /dev/shm

/dev/sda1       190M   33M  147M  19% /boot

n1:/pick        988M  1.4M  936M   1% /application/nginx-1.10.2/html/blog/wp-content/uploads

1.2.3 取消挂载

[[email protected] ~]# umount /application/nginx-1.10.2/html/blog/wp-content/uploads

[[email protected] ~]# df -h

Filesystem      Size  Used Avail Use% Mounted on

/dev/sda3       6.9G  3.7G  2.8G  58% /

tmpfs           491M     0  491M   0% /dev/shm

/dev/sda1       190M   33M  147M  19% /boot

1.2.4 停止卷

[[email protected] ~]# gluster volume stop pick

Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y

volume stop: pick: success

1.2.5 移除brick（必须移除再删除卷）

[[email protected] ~]# gluster volume remove-brick pick replica 1 n1:/data/  force

Removing brick(s) can result in data loss. Do you want to Continue? (y/n) y

volume remove-brick commit force: success

移除一个即可

1.2.6 删除卷

[[email protected] ~]# gluster volume delete pick

Deleting volume will erase all information about the volume. Do you want to continue? (y/n) y

volume delete: pick: success

1.2.7 移除池

[[email protected] ~]# gluster peer detach n2

peer detach: success

1.2.8 磁盘平衡

磁盘平衡时很有必要的(平衡布局:rebalance)，因为布局结构是静态的，当新的bricks加入先有卷，新创建的文件会分布到旧的Bricks中，只有平衡布局(rebalance)，新加入的磁盘才会生效。