基于Gluster分布式实现docker存储卷

Posted 2021-01-28 honeycomb

1.安装配置glusterfs
基础配置：
centos7 关闭firewalld/selinux
配置/etc/hosts和hostname

1 [[email protected] ~]# cat /etc/hosts
2 127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
3 ::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
4 192.168.138.130 glus1
5 192.168.138.187 glus2
6 192.168.138.188 glus3

安装glusterfs

1  yum -y install centos-release-gluster
2  yum --enable repo=centos-gluster*-test install glusterfs-server
3  yum --enablerepo=centos-gluster*-test install glusterfs-server
4  glusterfs -V
5  systemctl enable glusterd
6  systemctl start glusterd

准备存储盘

 1  fdisk /dev/sdb
 2  mkfs.xfs -i size=512 /dev/sdb1
 3  mkdir /gluster
 4  echo ‘/dev/sdb1 /gluster/ xfs defaults 1 2‘ >> /etc/fstab
 5  mount -a && mount
 6  在glus1
 7  gluster peer probe glus2
 8  gluster peer probe glus2
 9  systemctl status glusterd
10  gluster peer status
11  mkdir /gluster/docker

glusterfs 配置券

1 gluster volume create docker replica 3 glus1:/gluster/docker glus2:/gluster/docker glus3:/gluster/docker
2 gluster volume start docker
3 gluster volume set docker nfs.disable off
4 mount -t gluterfs glus1:/docker /mnt/mount -t glusterfs glus1:/docker /mnt/

测试 glusterfs 略

安装docker

1 sudo yum remove docker  docker-common docker-selinux docker-engine
2 sudo yum install -y yum-utils device-mapper-persistent-data lvm2
3 sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
4 yum list docker-ce --showduplicates | sort -r
5 yum -y install docker-ce.x86_64-17.03.0.ce
6 sudo yum install docker-ce-17.12.0.ce
7 systemctl start docker
8 systemctl enable docker

安装插件、配置测试

 1 docker plugin install sapk/plugin-gluster
 2 systemctl start docker
 3 docker plugin ls
 4 docker volume create --driver sapk/plugin-glusterv:1.0.7-3 --opt voluri="glus1,glus2,glus3:docker" --name houpj
 5 docker volume ls
 6 docker pull ubuntu
 7 [[email protected] ~]# docker volume create --driver sapk/plugin-gluster:v1.0.7-3 --opt voluri="glus1,glus2.glus3:docker" --name houpj
 8 [[email protected] ~]# docker volume ls
 9 DRIVER                         VOLUME NAME
10 sapk/plugin-gluster:v1.0.7-3   fuck
11 sapk/plugin-gluster:latest     houpj
12 [[email protected] ~]# docker run -v houpj:/mnt --rm -ti ubuntu
13 [email protected]:/# ls
14 bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
15 [email protected]:/mnt# echo 123 >tttt
16 [[email protected] ~]# cat /gluster/docker/tttt 
17 123
18 排查sapk/plugin-gluster一定要v1.0.7-3，latest 有bug 官方解释：https://github.com/sapk/docker-volume-gluster/issues/43