物理服务器Linux下软RAID和UUID方式挂载方法--Megacli64
Posted 莫等闲~
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了物理服务器Linux下软RAID和UUID方式挂载方法--Megacli64相关的知识，希望对你有一定的参考价值。
一、业务部门需求说明：公司最近来了一批服务器，用于大数据业务部署。数据节点服务器由14块物理磁盘，其中有2块是900G的盘，12块是4T的盘。在服务器系统安装时，进入系统的Bios界面：1）将2块900G的磁盘做成raid1用作系统盘；2）将其中的2块4T的磁盘做成raid1，分别挂载到/data1和/data2用作大数据日志存储；3）另外的10块4T的磁盘在系统安装时没做raid也没做分区，打算在系统安装后，登录到系统终端通过命令行进行直接进行10块盘的格式化，并分别挂载到/data3、/data4、/data5、/data6、/data7、/data8、/data9、/data10、/data11、/data12，用作大数据业务的数据盘，文件格式为ext4，采用uuid方式挂载，挂载属性为noatime,nobarrier。
待服务器系统安装后，登录机器，使用"fdisk -l"命令，发现除了看到4块做raid的盘，其余的10块物理磁盘看不到：
[[email protected] ~]# fdisk -l Disk /dev/sdb: 2001.1 GB, 2001111154688 bytes 255 heads, 63 sectors/track, 243287 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x0004a319 Device Boot Start End Blocks Id System /dev/sdb1 1 243288 1954208768 83 Linux Disk /dev/sdd: 4000.2 GB, 4000225165312 bytes 255 heads, 63 sectors/track, 486333 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/sdf: 4000.2 GB, 4000225165312 bytes 255 heads, 63 sectors/track, 486333 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/sde: 4000.2 GB, 4000225165312 bytes 255 heads, 63 sectors/track, 486333 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/sdh: 4000.2 GB, 4000225165312 bytes 255 heads, 63 sectors/track, 486333 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/sda: 899.5 GB, 899527213056 bytes 255 heads, 63 sectors/track, 109361 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00026ac4 Device Boot Start End Blocks Id System /dev/sda1 * 1 52 409600 83 Linux Partition 1 does not end on cylinder boundary. /dev/sda2 52 4229 33554432 82 Linux swap / Solaris /dev/sda3 4229 109362 844479488 83 Linux Disk /dev/sdc: 1999.1 GB, 1999114010624 bytes 255 heads, 63 sectors/track, 243045 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x0006d390 Device Boot Start End Blocks Id System /dev/sdc1 1 243046 1952258048 83 Linux
这时候，需要用到一款管理维护硬件RAID软件-MegaCli，它可以通过它来了解当前raid卡的所有信息，包括 raid卡的型号，raid的阵列类型，raid 上各磁盘状态，等等。操作记录如下：
1）下载及安装MegaCLI工具 下载地址：https://pan.baidu.com/s/1TAGHjTA19ZR8MGODaqy7Mg 提取密码：msbq 下载到/usr/loca/src目录下 [[email protected] ~]# cd /usr/local/src/ [[email protected] src]# ls ibm_utl_sraidmr_megacli-8.00.48_linux_32-64.zip [[email protected] src]# unzip ibm_utl_sraidmr_megacli-8.00.48_linux_32-64.zip [[email protected] src]# cd linux/ [[email protected] linux]# ls Lib_Utils-1.00-09.noarch.rpm MegaCli-8.00.48-1.i386.rpm [[email protected] linux]# rpm -ivh Lib_Utils-1.00-09.noarch.rpm MegaCli-8.00.48-1.i386.rpm 说明：安装完毕之后MegaCli64所在路径为/opt/MegaRAID/MegaCli/MegaCli64，在此路径下可以运行MegaCli64工具，切换到其它路径下则不能执行， 此时为了使用方便，可以考虑将/opt/MegaRAID/MegaCli/MegaCli64追加到系统PATH变量，或者像下面这样做（建议）： [[email protected] linux]# ln -s /opt/MegaRAID/MegaCli/MegaCli64 /bin/MegaCli64 [[email protected] linux]# ln -s /opt/MegaRAID/MegaCli/MegaCli64 /sbin/MegaCli64 2）使用MegaCli64命令进行相关操作 先查看磁盘数量。如下可以看出，一共有14块物理磁盘 [[email protected] linux]# MegaCli64 -PDList -aALL | egrep ‘Enclosure Device ID|Slot Number‘ Enclosure Device ID: 32 Slot Number: 0 Enclosure Device ID: 32 Slot Number: 1 Enclosure Device ID: 32 Slot Number: 2 Enclosure Device ID: 32 Slot Number: 3 Enclosure Device ID: 32 Slot Number: 4 Enclosure Device ID: 32 Slot Number: 5 Enclosure Device ID: 32 Slot Number: 6 Enclosure Device ID: 32 Slot Number: 7 Enclosure Device ID: 32 Slot Number: 8 Enclosure Device ID: 32 Slot Number: 9 Enclosure Device ID: 32 Slot Number: 10 Enclosure Device ID: 32 Slot Number: 11 Enclosure Device ID: 32 Slot Number: 12 Enclosure Device ID: 32 Slot Number: 13 接着查看磁盘信息 [[email protected] linux]# MegaCli64 -PDList -aALL Adapter #0 Enclosure Device ID: 32 Slot Number: 0 Enclosure position: 0 Device Id: 0 Sequence Number: 2 Media Error Count: 0 Other Error Count: 0 Predictive Failure Count: 0 Last Predictive Failure Event Seq Number: 0 PD Type: SATA Raw Size: 3.638 TB [0x1d1c0beb0 Sectors] Non Coerced Size: 3.637 TB [0x1d1b0beb0 Sectors] Coerced Size: 3.637 TB [0x1d1b00000 Sectors] Firmware state: Online, Spun Up SAS Address(0): 0x500056b3983fbac0 Connected Port Number: 0(path0) Inquiry Data: 4837K2DVF7DETOSHIBA MG04ACA400NY FK3D FDE Capable: Not Capable FDE Enable: Disable Secured: Unsecured Locked: Unlocked Needs EKM Attention: No Foreign State: None Device Speed: 6.0Gb/s Link Speed: 6.0Gb/s Media Type: Hard Disk Device Drive Temperature :30C (86.00 F) ........... ........... Enclosure Device ID: 32 Slot Number: 3 Enclosure position: 0 Device Id: 3 Sequence Number: 2 Media Error Count: 0 Other Error Count: 0 Predictive Failure Count: 0 Last Predictive Failure Event Seq Number: 0 PD Type: SATA Raw Size: 3.638 TB [0x1d1c0beb0 Sectors] Non Coerced Size: 3.637 TB [0x1d1b0beb0 Sectors] Coerced Size: 3.637 TB [0x1d1b00000 Sectors] Firmware state: unconfigured(good), Spun Up SAS Address(0): 0x500056b3983fbac3 Connected Port Number: 0(path0) Inquiry Data: 4838K1VCF7DETOSHIBA MG04ACA400NY FK3D FDE Capable: Not Capable FDE Enable: Disable Secured: Unsecured Locked: Unlocked Needs EKM Attention: No Foreign State: None Device Speed: 6.0Gb/s Link Speed: 6.0Gb/s Media Type: Hard Disk Device Drive Temperature :30C (86.00 F) ........... ........... 如上命令结果信息中，注意查看"Firmware state"的状态： 如果该状态为Online，就说明该磁盘已被做成raid阵列， 如果该状态为unconfigured(good)，就说明该磁盘没有被做成raid阵列，但状态OK。 由上面命令可知，前2块4T的磁盘做成了raid，其余10块4T的磁盘没有做raid，另2块900G的磁盘做成了raid。 -------------------------------------------------------------------------------------- 现在的做法是： 通过MegaCli工具将那10块4T的磁盘分别做成raid0，然后格式化成ext4，并使用uuid方式挂载。 依次对第3-12块盘做成raid0阵列，其中： -r0表示做成raid0阵列，[32:2]中的32为Enclosure Device ID，5为Slot Number。 WB Direct：磁盘Write back [[email protected] linux]# MegaCli64 -CfgLdAdd -r0[32:2] WB Direct -a0 [[email protected] linux]# MegaCli64 -CfgLdAdd -r0[32:3] WB Direct -a0 [[email protected] linux]# MegaCli64 -CfgLdAdd -r0[32:4] WB Direct -a0 [[email protected] linux]# MegaCli64 -CfgLdAdd -r0[32:5] WB Direct -a0 [[email protected] linux]# MegaCli64 -CfgLdAdd -r0[32:6] WB Direct -a0 [[email protected] linux]# MegaCli64 -CfgLdAdd -r0[32:7] WB Direct -a0 [[email protected] linux]# MegaCli64 -CfgLdAdd -r0[32:8] WB Direct -a0 [[email protected] linux]# MegaCli64 -CfgLdAdd -r0[32:9] WB Direct -a0 [[email protected] linux]# MegaCli64 -CfgLdAdd -r0[32:10] WB Direct -a0 [[email protected] linux]# MegaCli64 -CfgLdAdd -r0[32:11] WB Direct -a0 然后再次查看磁盘情况，发现"Firmware state"的状态都为Online，即所有磁盘都在raid阵列中了。 [[email protected] linux]# MegaCli64 -PDList -aALL 然后使用"fdisk -l"就能发现所有的物理磁盘了 [[email protected] linux]# fdisk -l Disk /dev/sdb: 1993.4 GB, 1993414541312 bytes 255 heads, 63 sectors/track, 242352 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x000cd8df Device Boot Start End Blocks Id System /dev/sdb1 1 242353 1946692608 83 Linux Disk /dev/sda: 899.5 GB, 899527213056 bytes 255 heads, 63 sectors/track, 109361 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x000b231b Device Boot Start End Blocks Id System /dev/sda1 * 1 52 409600 83 Linux Partition 1 does not end on cylinder boundary. /dev/sda2 52 4229 33554432 82 Linux swap / Solaris /dev/sda3 4229 109362 844479488 83 Linux Disk /dev/sdc: 2006.8 GB, 2006810624000 bytes 255 heads, 63 sectors/track, 243980 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x000010e5 Device Boot Start End Blocks Id System /dev/sdc1 1 243981 1959774208 83 Linux Disk /dev/sdd: 4000.2 GB, 4000225165312 bytes 255 heads, 63 sectors/track, 486333 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/sde: 4000.2 GB, 4000225165312 bytes 255 heads, 63 sectors/track, 486333 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/sdf: 4000.2 GB, 4000225165312 bytes 255 heads, 63 sectors/track, 486333 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/sdg: 4000.2 GB, 4000225165312 bytes 255 heads, 63 sectors/track, 486333 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/sdh: 4000.2 GB, 4000225165312 bytes 255 heads, 63 sectors/track, 486333 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/sdi: 4000.2 GB, 4000225165312 bytes 255 heads, 63 sectors/track, 486333 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/sdj: 4000.2 GB, 4000225165312 bytes 255 heads, 63 sectors/track, 486333 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/sdk: 4000.2 GB, 4000225165312 bytes 255 heads, 63 sectors/track, 486333 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/sdl: 4000.2 GB, 4000225165312 bytes 255 heads, 63 sectors/track, 486333 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/sdm: 4000.2 GB, 4000225165312 bytes 255 heads, 63 sectors/track, 486333 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 --------------------------------------------------------------------------- 对这10块磁盘进行格式化，脚本如下： [[email protected] ~]# cat disk.list /dev/sdd /dev/sde /dev/sdf /dev/sdg /dev/sdh /dev/sdi /dev/sdj /dev/sdk /dev/sdl /dev/sdm [[email protected] ~]# cat mkfs.disk.sh #!/bin/bash for i in `cat /root/disk.list` do echo ‘y‘ | /sbin/mkfs.ext4 $i & done [[email protected] ~]# sh -x mkfs.disk.sh 接着进行挂载 [[email protected] ~]# /bin/mkdir /data{3,4,5,6,7,8,9,10,11,12} 查看这10块盘的uuid（下面两种方式都可以） [[email protected] ~]# blkid [[email protected] ~]# ls -l /dev/disk/by-uuid/ [[email protected] ~]# blkid /dev/sda3: UUID="964bec23-58b4-4a6b-a96f-f2e3222fc096" TYPE="ext4" /dev/sdc1: UUID="9acdef9d-fbe1-4d9f-82ff-9e9920df868e" TYPE="ext4" /dev/sdb1: UUID="696f5971-4c7c-4312-a1c3-a20fc3772299" TYPE="ext4" /dev/sda1: UUID="ee26ded4-8334-4a0f-84bc-cc97d103714e" TYPE="ext4" /dev/sda2: UUID="316cb693-05fe-473d-a2ff-3c3c0e0e6c3d" TYPE="swap" /dev/sdd: UUID="f92e73be-526d-4d84-8f5b-95273ebbd352" TYPE="ext4" /dev/sde: UUID="0a6404ea-60dc-4e3e-b542-48a313e149dd" TYPE="ext4" /dev/sdf: UUID="05891dd0-256a-4f7f-a2de-f1f858eb2a95" TYPE="ext4" /dev/sdg: UUID="77df1f77-0168-430e-96a3-f2eb44e15242" TYPE="ext4" /dev/sdh: UUID="e1f11339-ad68-44a1-a600-066094439ed2" TYPE="ext4" /dev/sdi: UUID="628f1658-d8f9-4573-a124-0712b0c29e90" TYPE="ext4" /dev/sdj: UUID="9ee336b0-3960-4cfd-9cb6-c92535f45ebd" TYPE="ext4" /dev/sdk: UUID="bb6c1e2d-41b8-407d-b6df-df2e3ffc9c52" TYPE="ext4" /dev/sdl: UUID="9ca6aecf-e0f1-4338-a7eb-e8a1d2f3b017" TYPE="ext4" /dev/sdm: UUID="a5bf2880-4981-462a-8042-c6e913627c3d" TYPE="ext4" 单独找出这10块磁盘的uuid [[email protected] ~]# blkid|tail -10|awk ‘{print $2}‘ UUID="f92e73be-526d-4d84-8f5b-95273ebbd352" UUID="0a6404ea-60dc-4e3e-b542-48a313e149dd" UUID="05891dd0-256a-4f7f-a2de-f1f858eb2a95" UUID="77df1f77-0168-430e-96a3-f2eb44e15242" UUID="e1f11339-ad68-44a1-a600-066094439ed2" UUID="628f1658-d8f9-4573-a124-0712b0c29e90" UUID="9ee336b0-3960-4cfd-9cb6-c92535f45ebd" UUID="bb6c1e2d-41b8-407d-b6df-df2e3ffc9c52" UUID="9ca6aecf-e0f1-4338-a7eb-e8a1d2f3b017" UUID="a5bf2880-4981-462a-8042-c6e913627c3d" [[email protected] ~]# blkid|tail -10|awk ‘{print $2}‘|sed ‘s/"//g‘ UUID=f92e73be-526d-4d84-8f5b-95273ebbd352 UUID=0a6404ea-60dc-4e3e-b542-48a313e149dd UUID=05891dd0-256a-4f7f-a2de-f1f858eb2a95 UUID=77df1f77-0168-430e-96a3-f2eb44e15242 UUID=e1f11339-ad68-44a1-a600-066094439ed2 UUID=628f1658-d8f9-4573-a124-0712b0c29e90 UUID=9ee336b0-3960-4cfd-9cb6-c92535f45ebd UUID=bb6c1e2d-41b8-407d-b6df-df2e3ffc9c52 UUID=9ca6aecf-e0f1-4338-a7eb-e8a1d2f3b017 UUID=a5bf2880-4981-462a-8042-c6e913627c3d 将这10块磁盘的uuid好放到/etc/fastab开启启动文件里 [[email protected] ~]# cat /root/a.txt /data3 ext4 noatime,nobarrier 0 0 /data4 ext4 noatime,nobarrier 0 0 /data5 ext4 noatime,nobarrier 0 0 /data6 ext4 noatime,nobarrier 0 0 /data7 ext4 noatime,nobarrier 0 0 /data8 ext4 noatime,nobarrier 0 0 /data9 ext4 noatime,nobarrier 0 0 /data10 ext4 noatime,nobarrier 0 0 /data11 ext4 noatime,nobarrier 0 0 /data12 ext4 noatime,nobarrier 0 0 [[email protected] ~]# blkid|tail -10|awk ‘{print $2}‘|sed ‘s/"//g‘|paste - /root/a.txt >> /etc/fastab UUID=f92e73be-526d-4d84-8f5b-95273ebbd352 /data3 ext4 noatime,nobarrier 0 0 UUID=0a6404ea-60dc-4e3e-b542-48a313e149dd /data4 ext4 noatime,nobarrier 0 0 UUID=05891dd0-256a-4f7f-a2de-f1f858eb2a95 /data5 ext4 noatime,nobarrier 0 0 UUID=77df1f77-0168-430e-96a3-f2eb44e15242 /data6 ext4 noatime,nobarrier 0 0 UUID=e1f11339-ad68-44a1-a600-066094439ed2 /data7 ext4 noatime,nobarrier 0 0 UUID=628f1658-d8f9-4573-a124-0712b0c29e90 /data8 ext4 noatime,nobarrier 0 0 UUID=9ee336b0-3960-4cfd-9cb6-c92535f45ebd /data9 ext4 noatime,nobarrier 0 0 UUID=bb6c1e2d-41b8-407d-b6df-df2e3ffc9c52 /data10 ext4 noatime,nobarrier 0 0 UUID=9ca6aecf-e0f1-4338-a7eb-e8a1d2f3b017 /data11 ext4 noatime,nobarrier 0 0 UUID=a5bf2880-4981-462a-8042-c6e913627c3d /data12 ext4 noatime,nobarrier 0 0 最后将服务器重启，重启之后就能看到那10块磁盘的挂载情况
[[email protected] ~]# df -h Filesystem Size Used Avail Use% Mounted on /dev/sda3 793G 3.1G 750G 1% / tmpfs 63G 0 63G 0% /dev/shm /dev/sda1 380M 78M 282M 22% /boot /dev/sdb1 1.8T 68M 1.7T 1% /data1 /dev/sdc1 1.8T 68M 1.8T 1% /data2 /dev/sdd 3.6T 68M 3.4T 1% /data3 /dev/sde 3.6T 68M 3.4T 1% /data4 /dev/sdf 3.6T 68M 3.4T 1% /data5 /dev/sdg 3.6T 68M 3.4T 1% /data6 /dev/sdh 3.6T 68M 3.4T 1% /data7 /dev/sdi 3.6T 68M 3.4T 1% /data8 /dev/sdj 3.6T 68M 3.4T 1% /data9 /dev/sdk 3.6T 68M 3.4T 1% /data10 /dev/sdl 3.6T 68M 3.4T 1% /data11 /dev/sdm 3.6T 68M 3.4T 1% /data12 [[email protected] ~]# mount /dev/sda3 on / type ext4 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) tmpfs on /dev/shm type tmpfs (rw) /dev/sda1 on /boot type ext4 (rw) /dev/sdb1 on /data1 type ext4 (rw) /dev/sdc1 on /data2 type ext4 (rw) /dev/sdd on /data3 type ext4 (rw,noatime,nobarrier) /dev/sde on /data4 type ext4 (rw,noatime,nobarrier) /dev/sdf on /data5 type ext4 (rw,noatime,nobarrier) /dev/sdg on /data6 type ext4 (rw,noatime,nobarrier) /dev/sdh on /data7 type ext4 (rw,noatime,nobarrier) /dev/sdi on /data8 type ext4 (rw,noatime,nobarrier) /dev/sdj on /data9 type ext4 (rw,noatime,nobarrier) /dev/sdk on /data10 type ext4 (rw,noatime,nobarrier) /dev/sdl on /data11 type ext4 (rw,noatime,nobarrier) /dev/sdm on /data12 type ext4 (rw,noatime,nobarrier) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
以上是关于物理服务器Linux下软RAID和UUID方式挂载方法--Megacli64的主要内容，如果未能解决你的问题，请参考以下文章