linux之磁盘阵列实战
Posted
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了linux之磁盘阵列实战相关的知识，希望对你有一定的参考价值。
linux之磁盘阵列实战
一、磁盘阵列概述
磁盘阵列（Redundant Arrays of Independent Disks，RAID），有“独立磁盘构成的具有冗余能力的阵列”之意。是防止硬盘突然损坏导致数据丢失的一种冗余备份机制。
磁盘阵列有三种形式：外接式磁盘阵列柜、内接式磁盘阵列卡，利用软件来仿真。
二、常见磁盘阵列RAID类型：
1、RAID 0
RAID 0(也称条带模式)是组建磁盘阵列中最简单的一种形式，只需要2块以上的硬盘即可，成本低，可以提高整个磁盘的性能和吞吐量。RAID 0没有提供冗余或错误修复能力，但实现成本是最低的。如下图，如数据量为10G，第一块硬盘存储5G，第二块硬盘存储剩下的5G。当其中一块硬盘损坏时，数据就损坏了。生产环境不建议使用。
2、RAID 1
RAID 1(也称磁盘镜像)，是把一个磁盘的数据镜像到另一个磁盘上，也就是说数据在写入一块磁盘的同时，会在另一块闲置的磁盘上生成镜像文件。磁盘使用率为50%，其中一块磁盘损坏，不会影响数据，如下图。以四块80GB容量的硬盘来讲，可利用的磁盘空间仅为160GB。RAID 1多用在保存关键性的重要数据的场合。
3、RAID 5
RAID 5是分布式奇偶校验的独立磁盘结构。如下图，它的奇偶校验码存在于所有磁盘上，其中的p0代表第0带区的奇偶校验值，至少需要三块硬盘，第一块磁盘和第二块磁盘做一个异或运算得到一个校验位，每两块磁盘做一次异或运算，这样在一块磁盘损坏的时候，根据其余两块磁盘就可以推算出损坏的部分，从而保证了数据的完整性。磁盘使用率为（n-1）/ n ,工作中使用最多的就是raid 5的模式。
4、RAID 10
Raid 10是一个Raid 0与Raid1的组合体，它是利用奇偶校验实现条带集镜像，所以它继承了Raid0的快速和Raid1的安全，最少需要4块硬盘。如下图
三、各RAID类型实战
注：通过以上类型描述，以下均采用软RAID实现。创建软RAID命令为mdadm
mdadm命令常用参数如下：
-C或--creat：建立一个新阵列
-A：激活磁盘阵列
-D或--detail：打印阵列设备的详细内容
-s或--scan：扫描配置文件或/proc/mdstat得到阵列缺失信息
-f：将设备状态定为故障
-a或--add：添加设备到阵列
-v或--verbose：显示详细信息
-r：移除设备
-l或--level=：设定磁盘阵列的级别
-n或--raid-devices=：设定阵列成员(分区/磁盘)的数量
-x或--space-devices=：设定阵列中备用磁盘数量
-c或--chunk=：设定阵列的块chunk大小，单位为KB。chunk(块)，RAID的存储数据时每个数据段大小为4KB、64KB
-G或--grow：改变阵型大小或形态
1、RAID 0
示例要求：创建RAID 0；格式化并且永久挂载到指定目录。
示例环境：准备2块磁盘，/dev/sdb /dev/sdc，每一个硬盘上分出一个5G的分区，在/mnt目录下创建RAID0目录(mkdir -pv /mnt/RAID0,供挂载使用)
1)查看当前硬盘信息
[[email protected] ~]# fdisk -l Disk /dev/sda: 42.9 GB, 42949672960 bytes 255 heads, 63 sectors/track, 5221 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x000ecb54 Device Boot Start End Blocks Id System /dev/sda1 * 1 103 819200 83 Linux Partition 1 does not end on cylinder boundary. /dev/sda2 103 3927 30720000 8e Linux LVM Disk /dev/sdb: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/sdc: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000
2)为sdb硬盘分区,创建新主分区sdb1
[[email protected] ~]# fdisk /dev/sdb Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel with disk identifier 0x19ab7785. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won‘t be recoverable. Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) WARNING: DOS-compatible mode is deprecated. It‘s strongly recommended to switch off the mode (command ‘c‘) and change display units to sectors (command ‘u‘).
3)输入n回车新建分区，接着再输入p回车新建主分区
Command (m for help): n Command action e extended p primary partition (1-4) p
4)此处要求选择分区号在1-4间，输入1回车
Partition number (1-4): 1 First cylinder (1-1305, default 1): Using default value 1
5)设置分区分区大小为5G，输入+5G后回车
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305): +5G6)输入p查看分区是否成功，输入p回车，如下图：显示分区成功
Command (m for help): p Disk /dev/sdb: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x19ab7785 Device Boot Start End Blocks Id System /dev/sdb1 1 654 5253223+ 83 Linux
7)最后输入w回车，进行保存退出
Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks.
8)按上述方法，为硬盘sdc创建5G大小sdc1分区
Command (m for help): p Disk /dev/sdc: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x356d6b05 Device Boot Start End Blocks Id System /dev/sdc1 1 654 5253223+ 83 Linux
9)使用mdadm命令把sdb/sdc硬盘分别sdb1、sdc1分区创建为RAID 0
[[email protected] ~]# mdadm -C -v /dev/md0 -l 0 -n 2 /dev/sdb1 /dev/sdc1 mdadm: chunk size defaults to 512K mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md0 started. 注释：-C创建新磁盘阵列，-l 0 为RAID类型为0，2为硬盘个数。
10)查看阵列信息，命令mdadm -Ds，以及创建raid 0后/dev/md0硬盘分区大小信息
[[email protected] ~]# mdadm -Ds ARRAY /dev/md0 metadata=1.2 name=hc03:0 UUID=b524bc38:194dae34:41d4adec:24b3a69e [[email protected] ~]# fdisk -l /dev/md0 Disk /dev/md0: 10.8 GB, 10750001152 bytes 2 heads, 4 sectors/track, 2624512 cylinders Units = cylinders of 8 * 512 = 4096 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 524288 bytes / 1048576 bytes Disk identifier: 0x00000000
11)为/dev/md0新生成硬盘分区(与上述sdb或sdc分区相同)
[[email protected] ~]# fdisk /dev/md0 Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel with disk identifier 0xe58709bd. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won‘t be recoverable. Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) WARNING: DOS-compatible mode is deprecated. It‘s strongly recommended to switch off the mode (command ‘c‘) and change display units to sectors (command ‘u‘). Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-2624512, default 257): Using default value 257 Last cylinder, +cylinders or +size{K,M,G} (257-2624512, default 2624512): Using default value 2624512 Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks.
12)查看创建后的分区信息
[[email protected] ~]# fdisk -l /dev/md* last_lba(): I don‘t know how to handle files with mode 40755 Disk /dev/md0: 10.8 GB, 10750001152 bytes 2 heads, 4 sectors/track, 2624512 cylinders Units = cylinders of 8 * 512 = 4096 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 524288 bytes / 1048576 bytes Disk identifier: 0xe58709bd Device Boot Start End Blocks Id System /dev/md0p1 257 2624512 10497024 83 Linux Disk /dev/md0p1: 10.7 GB, 10748952576 bytes 2 heads, 4 sectors/track, 2624256 cylinders Units = cylinders of 8 * 512 = 4096 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 524288 bytes / 1048576 bytes Disk identifier: 0x00000000
13)格式化/dev/md0p1
[[email protected] ~]# mkfs.ext4 /dev/md0p1 mke2fs 1.41.12 (17-May-2010) Filesystem label= #文件系统标签 OS type: Linux #操作系统 Block size=4096 (log=2) #块大小 Fragment size=4096 (log=2) Stride=128 blocks, Stripe width=256 blocks 657072 inodes, 2624256 blocks 131212 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=2688548864 81 block groups 32768 blocks per group, 32768 fragments per group 8112 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 Writing inode tables: done #写入inode表 Creating journal (32768 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 29 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override.
14)设置开机启动自动挂载/dev/md0p1至/dev/RAID0目录
[[email protected] ~]# mount /dev/md0p1 /mnt/RAID0 [[email protected] ~]# mount -l /dev/mapper/my03-LogVol01 on / type ext4 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) tmpfs on /dev/shm type tmpfs (rw) /dev/sda1 on /boot type ext4 (rw) /dev/mapper/my03-LogVol02 on /usr type ext4 (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) /dev/md0p1 on /mnt/RAID0 type ext4 (rw) #挂载成功 [[email protected] ~]# cat /etc/fstab #加入开机启动自动挂载 # # /etc/fstab # Created by anaconda on Sat Oct 31 03:45:15 2015 # # Accessible filesystems, by reference, are maintained under ‘/dev/disk‘ # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info # /dev/mapper/my03-LogVol01 / ext4 defaults 1 1 UUID=c06d0732-24b8-4dfa-baba-f5ed54bea494 /boot ext4 defaults 1 2 /dev/mapper/my03-LogVol02 /usr ext4 defaults 1 2 /dev/mapper/my03-LogVol00 swap swap defaults 0 0 tmpfs /dev/shm tmpfs defaults 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 sysfs /sys sysfs defaults 0 0 proc /proc proc defaults 0 0 /dev/md0p1 /mnt/RAID0 ext4 defaults 0 0
2、RAID 1
示例要求：创建RAID 1；开机自动挂载至/mnt/RAID1并准备一个热备盘，模拟RAID 1中一个硬盘故障，并移除阵列。
示例环境：准备3块磁盘，/dev/sdb、/dev/sdc、/dev/sdd，每一个硬盘上分出一个2G的分区,分区以此/dev/sdb2、/dev/sdc2、/dev/sdd1
1)查看当前硬盘信息
[[email protected] ~]# fdisk -l Disk /dev/sda: 42.9 GB, 42949672960 bytes 255 heads, 63 sectors/track, 5221 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x000ecb54 Device Boot Start End Blocks Id System /dev/sda1 * 1 103 819200 83 Linux Partition 1 does not end on cylinder boundary. /dev/sda2 103 3927 30720000 8e Linux LVM Disk /dev/sdb: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x19ab7785 Device Boot Start End Blocks Id System /dev/sdb1 1 654 5253223+ 83 Linux Disk /dev/sdc: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x356d6b05 Device Boot Start End Blocks Id System /dev/sdc1 1 654 5253223+ 83 Linux Disk /dev/sdd: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000
2)为sdb硬盘分区,创建新主分区sdb2（创建方法同上）
[[email protected] ~]# fdisk /dev/sdb WARNING: DOS-compatible mode is deprecated. It‘s strongly recommended to switch off the mode (command ‘c‘) and change display units to sectors (command ‘u‘). Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 2 First cylinder (655-1305, default 655): Using default value 655 Last cylinder, +cylinders or +size{K,M,G} (655-1305, default 1305): +2G Command (m for help): P^H Disk /dev/sdb: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x19ab7785 Device Boot Start End Blocks Id System /dev/sdb1 1 654 5253223+ 83 Linux /dev/sdb2 655 916 2104515 83 Linux Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. WARNING: Re-reading the partition table failed with error 16: Device or resource busy. The kernel still uses the old table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8) Syncing disks.
注：建议重启，因为sdb1被作为RAID0并且挂载
3)按上述方法，为硬盘sdc创建2G大小sdc2分区，硬盘sdd创建2G大小sdd1分区
[[email protected] ~]# fdisk -l /dev/sdc Disk /dev/sdc: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x356d6b05 Device Boot Start End Blocks Id System /dev/sdc1 1 654 5253223+ 83 Linux /dev/sdc2 655 916 2104515 83 Linux [[email protected] ~]# fdisk /dev/sdd #创建sdd1分区 Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel with disk identifier 0x96e37a1b. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won‘t be recoverable. Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) WARNING: DOS-compatible mode is deprecated. It‘s strongly recommended to switch off the mode (command ‘c‘) and change display units to sectors (command ‘u‘). Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-1305, default 1): Using default value 1 Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305): +2G Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. [[email protected] ~]# fdisk -l /dev/sdd Disk /dev/sdd: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x96e37a1b Device Boot Start End Blocks Id System /dev/sdd1 1 262 2104483+ 83 Linux
4)使用mdadm命令把sdb/sdc硬盘分别sdb2、sdc2分区创建为RAID 2,sdd1为热备盘
[[email protected] ~]# mdadm -C -v /dev/md1 -l 1 -n 2 -x 1 /dev/sdb2 /dev/sdc2 /dev/sdd1 mdadm: Note: this array has metadata at the start and may not be suitable as a boot device. If you plan to store ‘/boot‘ on this device please ensure that your boot-loader understands md/v1.x metadata, or use --metadata=0.90 mdadm: size set to 2102400K Continue creating array? Continue creating array? (y/n) y mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md1 started. 注释：-C创建新磁盘阵列，-l 1 为RAID类型为1，2为硬盘个数，-x 1 为热备盘
5)查看阵列信息，命令mdadm -Ds，以及创建raid 0后/dev/md0硬盘分区大小信息
[[email protected] ~]# mdadm -Ds ARRAY /dev/md/hc03:0 metadata=1.2 name=hc03:0 UUID=b524bc38:194dae34:41d4adec:24b3a69e ARRAY /dev/md1 metadata=1.2 spares=1 name=hc03:1 UUID=ac6a9ef0:abadb228:f9c02876:1fad13c3
6)为/dev/md1新生成硬盘分区(与上述sdb或sdc分区相同)
[[email protected] ~]# fdisk /dev/md0 [[email protected] ~]# fdisk /dev/md1 Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel with disk identifier 0x34b7f63d. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won‘t be recoverable. Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) WARNING: DOS-compatible mode is deprecated. It‘s strongly recommended to switch off the mode (command ‘c‘) and change display units to sectors (command ‘u‘). Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-525600, default 1): Using default value 1 Last cylinder, +cylinders or +size{K,M,G} (1-525600, default 525600): Using default value 525600 Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks.
7)查看创建后的分区信息
[[email protected] ~]# fdisk -l /dev/md1* Disk /dev/md1: 2152 MB, 2152857600 bytes 2 heads, 4 sectors/track, 525600 cylinders Units = cylinders of 8 * 512 = 4096 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x34b7f63d Device Boot Start End Blocks Id System /dev/md1p1 1 525600 2102398 83 Linux Disk /dev/md127: 10.8 GB, 10750001152 bytes 2 heads, 4 sectors/track, 2624512 cylinders Units = cylinders of 8 * 512 = 4096 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 524288 bytes / 1048576 bytes Disk identifier: 0xe58709bd Device Boot Start End Blocks Id System /dev/md127p1 257 2624512 10497024 83 Linux Disk /dev/md127p1: 10.7 GB, 10748952576 bytes 2 heads, 4 sectors/track, 2624256 cylinders Units = cylinders of 8 * 512 = 4096 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 524288 bytes / 1048576 bytes Disk identifier: 0x00000000 Disk /dev/md1p1: 2152 MB, 2152855552 bytes 2 heads, 4 sectors/track, 525599 cylinders Units = cylinders of 8 * 512 = 4096 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000
8)格式化/dev/md1p1
[[email protected] ~]# mkfs.ext4 /dev/md1p1 mke2fs 1.41.12 (17-May-2010) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) Stride=0 blocks, Stripe width=0 blocks 131648 inodes, 525599 blocks 26279 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=541065216 17 block groups 32768 blocks per group, 32768 fragments per group 7744 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912 Writing inode tables: done Creating journal (16384 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 24 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override.
9)设置开机启动自动挂载/dev/md1p1至/mnt/RAID1目录
[[email protected] ~]# mkdir -pv /mnt/RAID1 mkdir: created directory `/mnt/RAID1‘ [[email protected] ~]# mount /dev/md1p1 /mnt/RAID1 [[email protected] ~]# mount -l /dev/mapper/my03-LogVol01 on / type ext4 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) tmpfs on /dev/shm type tmpfs (rw) /dev/sda1 on /boot type ext4 (rw) /dev/mapper/my03-LogVol02 on /usr type ext4 (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) /dev/md1p1 on /mnt/RAID1 type ext4 (rw) #挂载成功 [[email protected] ~]# cat /etc/fstab # # /etc/fstab # Created by anaconda on Sat Oct 31 03:45:15 2015 # # Accessible filesystems, by reference, are maintained under ‘/dev/disk‘ # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info # /dev/mapper/my03-LogVol01 / ext4 defaults 1 1 UUID=c06d0732-24b8-4dfa-baba-f5ed54bea494 /boot ext4 defaults 1 2 /dev/mapper/my03-LogVol02 /usr ext4 defaults 1 2 /dev/mapper/my03-LogVol00 swap swap defaults 0 0 tmpfs /dev/shm tmpfs defaults 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 sysfs /sys sysfs defaults 0 0 proc /proc proc defaults 0 0 /dev/md0p1 /mnt/RAID0 ext4 defaults 0 0 /dev/md1p1 /mnt/RAID1 ext4 defaults 0 0
10)使用/proc/mdstat查看阵列信息
[[email protected] ~]# cat /proc/mdstat Personalities : [raid0] [raid1] md1 : active raid1 sdd1[2](S) sdc2[1] sdb2[0] 2102400 blocks super 1.2 [2/2] [UU] md127 : active raid0 sdb1[0] sdc1[1] 10498048 blocks super 1.2 512k chunks unused devices: <none> 注释： #sdd1[2](s):其中s表示备用盘 #[2/2]表示raid 1中的两块磁盘都是正常的 #active表示此阵列正常读写
11)生成mdadm.conf配置文件
[[email protected] ~]# mdadm -Ds > /etc/mdadm.conf [[email protected] ~]# cat /etc/mdadm.conf ARRAY /dev/md/hc03:0 metadata=1.2 name=hc03:0 UUID=b524bc38:194dae34:41d4adec:24b3a69e ARRAY /dev/md1 metadata=1.2 spares=1 name=hc03:1 UUID=ac6a9ef0:abadb228:f9c02876:1fad13c3 注释： #spares=1 存在一个热备盘
12)模拟/dev/sdc2为故障盘，重启一个终端动态刷新/proc/mdstat配置信息，命令：watch -n 1 cat /proc/mdstat
[[email protected] ~]# mdadm -f /dev/md1 /dev/sdc2 #设定sdc2为故障 mdadm: set /dev/sdc2 faulty in /dev/md1 [[email protected] ~]# cat /proc/mdstat Personalities : [raid0] [raid1] md1 : active raid1 sdd1[2] sdc2[1](F) sdb2[0] 2102400 blocks super 1.2 [2/2] [UU] md127 : active raid0 sdb1[0] sdc1[1] 10498048 blocks super 1.2 512k chunks unused devices: <none> 注释： sdc2[1](F) 为故障 sdd1[2]并且已经接替故障盘
13)移除故障盘sdc2，并且更新/etc/mdadm.conf配置文件
[[email protected] ~]# mdadm -r /dev/md1 /dev/sdc2 mdadm: hot removed /dev/sdc2 from /dev/md1 [[email protected] ~]# cat /proc/mdstat Personalities : [raid0] [raid1] md1 : active raid1 sdd1[2] sdb2[0] 2102400 blocks super 1.2 [2/2] [UU] md127 : active raid0 sdb1[0] sdc1[1] 10498048 blocks super 1.2 512k chunks unused devices: <none> [[email protected] ~]# mdadm -Ds > /etc/mdadm.conf
3、RAID 5
示例要求：创建RAID 5；开机自动挂载至/mnt/RAID5并准备一个热备盘，模拟RAID 5中一个硬盘故障，并移除阵列，同时再扩展硬盘。
示例环境：准备5块磁盘，由于资源比较充裕，直接添加新的盘/dev/sdc、/dev/sdd、/dev/sde、/dev/sdf、/dev/sdg，每一个硬盘上分出一个2G的分区,分区以此/dev/sdc1、/dev/sdd1、/dev/sde1、/dev/sdf1、/dev/sdg1
1)创建以上新分区方法，请参照上述相关操作完成，这里不再阐述！
Device Boot Start End Blocks Id System /dev/sdc1 1 262 2104483+ 83 Linux Device Boot Start End Blocks Id System /dev/sdd1 1 262 2104483+ 83 Linux Device Boot Start End Blocks Id System /dev/sde1 1 262 2104483+ 83 Linux Device Boot Start End Blocks Id System /dev/sdf1 1 262 2104483+ 83 Linux Device Boot Start End Blocks Id System /dev/sdg1 1 262 2104483+ 83 Linux
2)使用mdadm命令把sdc/sdd/sde/sdf硬盘分别sdc1/sdd1/sde1分区创建为RAID 5,sdf1为热备盘
[[email protected] ~]# mdadm -C -v /dev/md5 -l 5 -n 3 -x 1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 mdadm: layout defaults to left-symmetric mdadm: layout defaults to left-symmetric mdadm: chunk size defaults to 512K mdadm: size set to 2102272K mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md5 started.
注释：-C创建新磁盘阵列，-l 5 为RAID类型为5，3为硬盘个数，-x 1 为热备盘
3)查看阵列信息，命令mdadm -Ds，以及创建raid 0后/dev/md0硬盘分区大小信息，并生成配置文件/etc/mdadm.conf
[[email protected] ~]# mdadm -Ds > /etc/mdadm.conf ARRAY /dev/md5 metadata=1.2 spares=1 name=lvs:5 UUID=8a62a5ad:9b1fd7ae:09b5f7e2:c0756e62
4)使用/proc/mdstat查看阵列信息
[[email protected] ~]# cat /proc/mdstat Personalities : [raid6] [raid5] [raid4] md5 : active raid5 sde1[4] sdf1[3](S) sdd1[1] sdc1[0] 4204544 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU] 注释： #sdf1[3](s):其中s表示备用盘 #[3/3]表示raid 1中的两块磁盘都是正常的 #active表示此阵列正常读写
5)模拟/dev/sde1为故障盘，重启一个终端动态刷新/proc/mdstat配置信息，命令：watch -n 1 cat /proc/mdstat
[[email protected] ~]# mdadm -f /dev/md5 /dev/sde1 #设定sde1为故障 [[email protected] ~]# mdadm -f /dev/md5 /dev/sde1 mdadm: set /dev/sde1 faulty in /dev/md5 [[email protected] ~]# cat /proc/mdstat Personalities : [raid6] [raid5] [raid4] md5 : active raid5 sde1[4](F) sdf1[3] sdd1[1] sdc1[0] 4204544 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU] unused devices: <none> 注释： sde1[4](F) 为故障 sdf1[3]并且已经接替故障盘 6)移除故障盘sde1,更新/etc/mdadm.conf配置文件
[[email protected] ~]# mdadm -r /dev/md5 /dev/sde1 mdadm: hot removed /dev/sde1 from /dev/md5 [[email protected] ~]# cat /proc/mdstat Personalities : [raid6] [raid5] [raid4] md5 : active raid5 sdf1[3] sdd1[1] sdc1[0] 4204544 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU] unused devices: <none> [[email protected] ~]# mdadm -Ds > /etc/mdadm.conf
7)md5磁盘分区格式化，并开机自动挂载至/mnt/RAID5目录
[[email protected] ~]# mkdir -pv /mnt/RAID5 #创建/mnt/RAID5目录 mkdir: 已创建目录 "/mnt/RAID5" [[email protected] ~]# fdisk /dev/md5 #磁盘分区 Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel with disk identifier 0xc24c42ef. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won‘t be recoverable. Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) WARNING: DOS-compatible mode is deprecated. It‘s strongly recommended to switch off the mode (command ‘c‘) and change display units to sectors (command ‘u‘). Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-1051136, default 257): Using default value 257 Last cylinder, +cylinders or +size{K,M,G} (257-1051136, default 1051136): Using default value 1051136 Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. [[email protected] ~]# mkfs.ext4 /dev/md5p1 #格式化分区 mke2fs 1.41.12 (17-May-2010) 文件系统标签= 操作系统:Linux 块大小=4096 (log=2) 分块大小=4096 (log=2) Stride=128 blocks, Stripe width=256 blocks 262944 inodes, 1050880 blocks 52544 blocks (5.00%) reserved for the super user 第一个数据块=0 Maximum filesystem blocks=1077936128 33 block groups 32768 blocks per group, 32768 fragments per group 7968 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736 正在写入inode表: 完成 Creating journal (32768 blocks): 完成 Writing superblocks and filesystem accounting information: 完成 This filesystem will be automatically checked every 39 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. [[email protected] ~]# mount /dev/md5p1 /mnt/RAID5 #挂载 [[email protected] ~]# mount -l #查看挂载 /dev/mapper/vg_lvs-lv_root on / type ext4 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) tmpfs on /dev/shm type tmpfs (rw) /dev/sda1 on /boot type ext4 (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) /dev/md5p1 on /mnt/RAID5 type ext4 (rw) #正常挂载 [[email protected] ~]# cat /etc/fstab #添加至开机启动挂载 # # /etc/fstab # Created by anaconda on Mon Nov 2 18:20:45 2015 # # Accessible filesystems, by reference, are maintained under ‘/dev/disk‘ # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info # /dev/mapper/vg_lvs-lv_root / ext4 defaults 1 1 UUID=146bb43b-343e-4f5a-90c6-4056c695bfa1 /boot ext4 defaults 1 2 /dev/mapper/vg_lvs-lv_swap swap swap defaults 0 0 tmpfs /dev/shm tmpfs defaults 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 sysfs /sys sysfs defaults 0 0 proc /proc proc defaults 0 0 /dev/md5p1 /mnt/RAID5 ext4 defaults 0 0
8)操作停止和激活阵列，使用mdadm -S 或 mdadm -As(注意：挂载中的阵列不允许停止，需要卸载挂载)
如：停止上面阵列RAID5
[[email protected] ~]# mdadm -S /dev/md5 mdadm: Cannot get exclusive access to /dev/md5:Perhaps a running process, mounted filesystem or active volume group? [[email protected] ~]# umount /dev/md5p1 /mnt/RAID5 umount: /mnt/RAID5: not mounted [[email protected] ~]# mdadm -S /dev/md5 mdadm: stopped /dev/md5 注：mdadm -Ss 为停止所有阵列
激活停止的RAID5，需要重新挂载
激活raid 5阵列：
[[email protected] ~]# mdadm -As mdadm: /dev/md/5 has been started with 3 drives. mdadm: Found some drive for an array that is already active: /dev/md/5 mdadm: giving up. [[email protected] ~]# mount /dev/md/5p1 /mnt/RAID5 #重新挂载
9)上面模拟一个硬盘故障，如何向阵列中添加一个磁盘，这里以另外一个磁盘/dev/sdg1为例(注意：挂载中的队列不允许，添加磁盘)
[[email protected] ~]# umount /dev/md/5p1 /mnt/RAID5 #卸载挂载 [[email protected] ~]# mdadm -a /dev/md5 /dev/sdg1 mdadm: added /dev/sdg1 [[email protected] ~]# cat /proc/mdstat Personalities : [raid6] [raid5] [raid4] md5 : active raid5 sdg1[4](S) sdc1[0] sdf1[3] sdd1[1] 4204544 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU] unused devices: <none> 注： sdg1[4](S) #说明sdg1已经是备用盘了 [[email protected] ~]# mdadm -G /dev/md5 -n 4 #添加至md5 [[email protected] ~]# cat /proc/mdstat Personalities : [raid6] [raid5] [raid4] md5 : active raid5 sdg1[4] sdc1[0] sdf1[3] sdd1[1] 4204544 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/4] [UUUU] [=======>.............] reshape = 36.0% (757580/2102272) finish=0.2min speed=108225K/sec unused devices: <none> 注： active raid5 sdg1[4] sdc1[0] sdf1[3] sdd1[1] #当前活动的已经有4个盘了 [[email protected] ~]# mdadm -Ds > /etc/mdadm.conf #刷新配置 [[email protected] ~]# mdadm -Ds ARRAY /dev/md/5 metadata=1.2 name=lvs:5 UUID=8a62a5ad:9b1fd7ae:09b5f7e2:c0756e62
4、RAID 10
示例要求：创建RAID 10；开机自动挂载至/mnt/RAID10。
示例环境：准备4块磁盘，直接添加新的盘/dev/sdb、/dev/sdc、/dev/sdd、/dev/sde，每一个硬盘上分出一个5G的分区,分区以此/dev/sdb1、/dev/sdc1、/dev/sdd1、/dev/sde1。
1)创建以上新分区方法，请参照上述相关操作完成，这里不再阐述！
Device Boot Start End Blocks Id System /dev/sdb1 1 652 5237158+ 83 Linux Device Boot Start End Blocks Id System /dev/sdc1 1 652 5237158+ 83 Linux Device Boot Start End Blocks Id System /dev/sdd1 1 652 5237158+ 83 Linux Device Boot Start End Blocks Id System /dev/sde1 1 652 5237158+ 83 Linux
2)使用mdadm命令把sdb/sdc和/sdd/sde硬盘分别sdb1/sdc1和/sdd1/sde1分区分别两个创建为RAID 1
[[email protected] ~]# mdadm -C -v /dev/md00 -l 1 -n 2 /dev/sdb1 /dev/sdc1 mdadm: Note: this array has metadata at the start and may not be suitable as a boot device. If you plan to store ‘/boot‘ on this device please ensure that your boot-loader understands md/v1.x metadata, or use --metadata=0.90 mdadm: size set to 5233024K Continue creating array? Continue creating array? (y/n) y mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md00 started. [[email protected] ~]# mdadm -C -v /dev/md01 -l 1 -n 2 /dev/sdd1 /dev/sde1 mdadm: Note: this array has metadata at the start and may not be suitable as a boot device. If you plan to store ‘/boot‘ on this device please ensure that your boot-loader understands md/v1.x metadata, or use --metadata=0.90 mdadm: size set to 5233024K Continue creating array? Continue creating array? (y/n) y mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md01 started. 注释：-C创建新磁盘阵列，-l 1 为RAID类型为1，2为硬盘个数
3)把上面创建新/dev/md00和/dev/md01创建为raid 0
[[email protected] ~]# mdadm -C -v /dev/md10 -l 0 -n 2 /dev/md0 /dev/md1 mdadm: chunk size defaults to 512K mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md10 started.
注意：上面创建用新的磁盘/dev/md00、/dev/md01用做raid 1 之后，名称变为/dev/md0和/dev/md1(可用fdisk -l 查看)
4)查看阵列信息，命令mdadm -Ds或/proc/mdstat，并生成配置文件/etc/mdadm.conf
[[email protected] ~]# cat /proc/mdstat Personalities : [raid1] [raid0] md10 : active raid0 md1[1] md0[0] 10457088 blocks super 1.2 512k chunks md1 : active raid1 sde1[1] sdd1[0] 5233024 blocks super 1.2 [2/2] [UU] md0 : active raid1 sdc1[1] sdb1[0] 5233024 blocks super 1.2 [2/2] [UU] unused devices: <none> 注释： #sdc1[1] sdb1[0]和sde1[1] sdd1[0]:为RAID 1 #md1[1] 和 md0[0]:为RAID 0 [[email protected] ~]# mdadm -Ds > /etc/mdadm.conf ARRAY /dev/md0 metadata=1.2 name=node1:00 UUID=dc9713a3:487f0925:0b8dcd0f:d13aec68 ARRAY /dev/md1 metadata=1.2 name=node1:01 UUID=edce41b0:6bb6b1cc:de7233fb:41cc27fb ARRAY /dev/md10 metadata=1.2 name=node1:10 UUID=58b21c66:8a8d9b01:c3900b13:6d2d5292
5)md10创建后进行磁盘分区格式化，并开机自动挂载至目录(这里不再演示，请自行参照上述操作)
5、如何删除RAID 阵列，这里上面的raid 10(md10)为例子
1)取消挂载 umount /dev/md10p1(如/dev/md10有挂载)
2)停止raid 10设备
[[email protected] ~]# mdadm -S /dev/md10 mdadm: stopped /dev/md10
3)删除/etc/mdadm.conf文件
[[email protected] ~]# rm -rf /etc/mdadm.conf
4)清除磁盘/dev/md0和/dev/md1的raid标识
[[email protected] ~]# mdadm --misc --zero-superblock /dev/md0 [[email protected] ~]# mdadm --misc --zero-superblock /dev/md1 注释： --misc #修改MD相关设备 --zero-superblock #擦除设备中MD的超级块
[[email protected] ~]# mdadm -Ds #查看 ARRAY /dev/md0 metadata=1.2 name=node1:00 UUID=dc9713a3:487f0925:0b8dcd0f:d13aec68 ARRAY /dev/md1 metadata=1.2 name=node1:01 UUID=edce41b0:6bb6b1cc:de7233fb:41cc27fb [[email protected] ~]# cat /proc/mdstat #查看当前阵列 Personalities : [raid1] [raid0] md1 : active raid1 sde1[1] sdd1[0] 5233024 blocks super 1.2 [2/2] [UU] md0 : active raid1 sdc1[1] sdb1[0] 5233024 blocks super 1.2 [2/2] [UU] unused devices: <none>
本文出自 “一万小时定律” 博客，请务必保留此出处http://daisywei.blog.51cto.com/7837970/1879561
以上是关于linux之磁盘阵列实战的主要内容，如果未能解决你的问题，请参考以下文章
Linux系统KVM虚拟机实战LVM逻辑卷之扩展Swap交换分区