22_2020年最新部署Ceph集群 Ceph块存储

Posted 2021-03-15 luwei0915

1. 下载ceph nautilus 版本yum源
   地址：https://mirrors.aliyun.com/ceph/rpm-nautilus/el7/
   下载三个文件夹里对应 14.2.6-0.el7 的 rpm
      aarch64/    14-Jan-2020 23:20
      noarch/     14-Jan-2020 23:21
      x86_64/     14-Jan-2020 23:24
1.1 下载aarch64文件夹对应版本的rpm文件：(物理机)
]# mkdir rpm
]# cd rpm/
rpm]# vim aarch64.txt
注意：用鼠标全选复制网页：
     "https://mirrors.aliyun.com/ceph/rpm-nautilus/el7/aarch64/"
      上面所有的文字粘贴到aarch.txt
如下图：

1.2 编写脚本：
rpm]# cat get.sh
#!/bin/bash
rpm_file=/root/rpm/$1.txt
rpm_netaddr=https://mirrors.aliyun.com/ceph/rpm-nautilus/el7/$1
for i in `cat $rpm_file`
do
    if [[ $i =~ rpm ]] && [[ $i =~ 14.2.6-0 ]]
    then
        wget $rpm_netaddr/$i
    fi
done

1.3 执行脚本，下载rpm文件：
rpm]# bash get.sh aarch64
查看:
rpm]# ls
aarch64.txt                                    libradospp-devel-14.2.6-0.el7.aarch64.rpm
ceph-14.2.6-0.el7.aarch64.rpm                  libradosstriper1-14.2.6-0.el7.aarch64.rpm
ceph-base-14.2.6-0.el7.aarch64.rpm             libradosstriper-devel-14.2.6-0.el7.aarch64.rpm
ceph-common-14.2.6-0.el7.aarch64.rpm           librbd1-14.2.6-0.el7.aarch64.rpm
ceph-debuginfo-14.2.6-0.el7.aarch64.rpm        librbd-devel-14.2.6-0.el7.aarch64.rpm
cephfs-java-14.2.6-0.el7.aarch64.rpm           librgw2-14.2.6-0.el7.aarch64.rpm
ceph-fuse-14.2.6-0.el7.aarch64.rpm             librgw-devel-14.2.6-0.el7.aarch64.rpm
ceph-mds-14.2.6-0.el7.aarch64.rpm              python3-ceph-argparse-14.2.6-0.el7.aarch64.rpm
ceph-mgr-14.2.6-0.el7.aarch64.rpm              python3-cephfs-14.2.6-0.el7.aarch64.rpm
ceph-mon-14.2.6-0.el7.aarch64.rpm              python3-rados-14.2.6-0.el7.aarch64.rpm
ceph-osd-14.2.6-0.el7.aarch64.rpm              python3-rbd-14.2.6-0.el7.aarch64.rpm
ceph-radosgw-14.2.6-0.el7.aarch64.rpm          python3-rgw-14.2.6-0.el7.aarch64.rpm
ceph-resource-agents-14.2.6-0.el7.aarch64.rpm  python-ceph-argparse-14.2.6-0.el7.aarch64.rpm
ceph-selinux-14.2.6-0.el7.aarch64.rpm          python-ceph-compat-14.2.6-0.el7.aarch64.rpm
ceph-test-14.2.6-0.el7.aarch64.rpm             python-cephfs-14.2.6-0.el7.aarch64.rpm
get.sh                                         python-rados-14.2.6-0.el7.aarch64.rpm
libcephfs2-14.2.6-0.el7.aarch64.rpm            python-rbd-14.2.6-0.el7.aarch64.rpm
libcephfs-devel-14.2.6-0.el7.aarch64.rpm       python-rgw-14.2.6-0.el7.aarch64.rpm
libcephfs_jni1-14.2.6-0.el7.aarch64.rpm        rados-objclass-devel-14.2.6-0.el7.aarch64.rpm
libcephfs_jni-devel-14.2.6-0.el7.aarch64.rpm   rbd-fuse-14.2.6-0.el7.aarch64.rpm
librados2-14.2.6-0.el7.aarch64.rpm             rbd-mirror-14.2.6-0.el7.aarch64.rpm
librados-devel-14.2.6-0.el7.aarch64.rpm        rbd-nbd-14.2.6-0.el7.aarch64.rpm

1.4 如法炮制，修改aarch.txt文件名和内容，下载noarch/ 和 x86_64/ 两个文件夹里面的 14.2.6的rpm文件。

1.5 注意
noarch/ 文件夹下，有些rpm文件并未显示全名，例如：
  ceph-mgr-diskprediction-cloud-14.2.6-0.el7.noar..> 14-Jan-2020 23:18  85684
   脚本下载不到，此时需要手动点击链接下载。
   同时，需要手动下载：（别问为啥)
  ceph-deploy-2.0.1-0.noarch.rpm
  ceph-medic-1.0.4-16.g60cf7e9.el7.noarch.rpm
  ceph-release-1-1.el7.noarch.rpm

1.6 将下载的rpm文件制作本地yum源，给虚拟机ceph集群使用
]# mv /root/rpm /var/ftp/pub/ceph
]# cd /var/ftp/pub/ceph/
ceph]# createrepo .
Spawning worker 0 with 24 pkgs
Spawning worker 1 with 24 pkgs
Spawning worker 2 with 23 pkgs
Spawning worker 3 with 23 pkgs
Workers Finished
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete

]# vim /etc/yum.repos.d/ceph.repo
[ceph]
name=ceph repo
baseurl=ftp://192.168.4.1/pub/ceph
gpgcheck=0
enable=1

然后将此文件发到各个虚拟机即可。

2. 创建虚拟机，准备集群环境
2.1 创建虚拟机，修改主机名，启动网卡
lient :192.168.4.10
node1 :192.168.4.11
ndoe2 :192.168.4.12
node3 :192.168.4.13

2.2 配置四台机器互相远程无密码连接（包括自己）
]# ssh-keygen -f /root/.ssh/id_rsa -N ‘‘
]# for i in 10 11 12 13
> do
> ssh-copy-id 192.168.4.$i
> done

2.3 修改/etc/hosts并同步到所有主机。
警告：/etc/hosts解析的域名必须与本机主机名一致！
]# vim /etc/hosts
... ...
192.168.4.10  client
192.168.4.11  node1
192.168.4.12  node2
192.168.4.13  node3

2.3 配置NTP时间同步
真实物理机创建NTP服务器
]# yum -y install chrony
]# vim /etc/chrony.conf
server ntp.aliyun.com iburst
allow 192.168.4.0/24
local stratum 10
]# systemctl restart chronyd
]# chronyc sources -v        #出现*时间同步成功
...
^* 203.107.6.88...

其他所有节点与NTP服务器同步时间（以node1为例）。
]# vim /etc/chrony.conf
server 192.168.4.1 iburst
]# systemctl restart chronyd
]# chronyc sources -v        #出现*时间同步成功

2.4 准备存储磁盘
物理机上为每个虚拟机准备3块磁盘
]# lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
vdb    252:16   0   20G  0 disk
vdc    252:32   0   20G  0 disk
vdd    252:48   0   20G  0 disk

未完待续...