安装oracle11grac时使用asmlib和udev的区别
Posted
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了安装oracle11grac时使用asmlib和udev的区别相关的知识，希望对你有一定的参考价值。
参考技术A 利用UDEV服务解决RAC ASM存储设备名1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
在<Why ASMLIB and why not?>我们介绍了使用ASMLIB作为一种专门为Oracle Automatic Storage Management特性设计的内核支持库(kernel support library)的优缺点，同时建议使用成熟的UDEV方案来替代ASMLIB。
这里我们就给出配置UDEV的具体步骤，还是比较简单的:
1.确认在所有RAC节点上已经安装了必要的UDEV包
[root@rh2 ~]# rpm -qa|grep udev
udev-095-14.21.el5
2.通过scsi_id获取设备的块设备的唯一标识名，假设系统上已有LUN sdc-sdp
for i in c d e f g h i j k l m n o p ;
do
echo "sd$i" "`scsi_id -g -u -s /block/sd$i` ";
done
sdc 1IET_00010001
sdd 1IET_00010002
sde 1IET_00010003
sdf 1IET_00010004
sdg 1IET_00010005
sdh 1IET_00010006
sdi 1IET_00010007
sdj 1IET_00010008
sdk 1IET_00010009
sdl 1IET_0001000a
sdm 1IET_0001000b
sdn 1IET_0001000c
sdo 1IET_0001000d
sdp 1IET_0001000e
以上列出于块设备名对应的唯一标识名
3.创建必要的UDEV配置文件，
首先切换到配置文件目录
[root@rh2 ~]# cd /etc/udev/rules.d
定义必要的规则配置文件
[root@rh2 rules.d]# touch 99-oracle-asmdevices.rules
[root@rh2 rules.d]# cat 99-oracle-asmdevices.rules
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="1IET_00010001", NAME="ocr1", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="1IET_00010002", NAME="ocr2", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="1IET_00010003", NAME="asm-disk1", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="1IET_00010004", NAME="asm-disk2", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="1IET_00010005", NAME="asm-disk3", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="1IET_00010006", NAME="asm-disk4", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="1IET_00010007", NAME="asm-disk5", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="1IET_00010008", NAME="asm-disk6", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="1IET_00010009", NAME="asm-disk7", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="1IET_0001000a", NAME="asm-disk8", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="1IET_0001000b", NAME="asm-disk9", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="1IET_0001000c", NAME="asm-disk10", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="1IET_0001000d", NAME="asm-disk11", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="1IET_0001000e", NAME="asm-disk12", OWNER="grid", GROUP="asmadmin", MODE="0660"
Result 为/sbin/scsi_id -g -u -s %p的输出--Match the returned string of the last PROGRAM call. This key may be
used in any following rule after a PROGRAM call.
按顺序填入刚才获取的唯一标识名即可
OWNER为安装Grid Infrastructure的用户，在11gr2中一般为grid，GROUP为asmadmin
MODE采用0660即可
NAME为UDEV映射后的设备名，
建议为OCR和VOTE DISK创建独立的DISKGROUP，为了容易区分将该DISKGROUP专用的设备命名为ocr1..ocrn的形式
其余磁盘可以根据其实际用途或磁盘组名来命名
4.将该规则文件拷贝到其他节点上
[root@rh2 rules.d]# scp 99-oracle-asmdevices.rules Other_node:/etc/udev/rules.d
5.在所有节点上启动udev服务，或者重启服务器即可
[root@rh2 rules.d]# /sbin/udevcontrol reload_rules
[root@rh2 rules.d]# /sbin/start_udev
Starting udev: [ OK ]
6.检查设备是否到位
[root@rh2 rules.d]# cd /dev
[root@rh2 dev]# <a href="https://www.baidu.com/s?wd=ls&tn=44039180_cpr&fenlei=mv6quAkxTZn0IZRqIHckPjm4nH00T1Y4PvD3PHKBujb4njD1P1uW0ZwV5Hcvrjm3rH6sPfKWUMw85HfYnjn4nH6sgvPsT6KdThsqpZwYTjCEQLGCpyw9Uz4Bmy-bIi4WUvYETgN-TLwGUv3ErHD3njfLrHTvnjnYP1fYPWR4" target="_blank" class="baidu-highlight">ls</a> <a href="https://www.baidu.com/s?wd=-l&tn=44039180_cpr&fenlei=mv6quAkxTZn0IZRqIHckPjm4nH00T1Y4PvD3PHKBujb4njD1P1uW0ZwV5Hcvrjm3rH6sPfKWUMw85HfYnjn4nH6sgvPsT6KdThsqpZwYTjCEQLGCpyw9Uz4Bmy-bIi4WUvYETgN-TLwGUv3ErHD3njfLrHTvnjnYP1fYPWR4" target="_blank" class="baidu-highlight">-l</a> ocr*
brw-rw---- 1 grid asmadmin 8, 32 Jul 10 17:31 ocr1
brw-rw---- 1 grid asmadmin 8, 48 Jul 10 17:31 ocr2
[root@rh2 dev]# <a href="https://www.baidu.com/s?wd=ls&tn=44039180_cpr&fenlei=mv6quAkxTZn0IZRqIHckPjm4nH00T1Y4PvD3PHKBujb4njD1P1uW0ZwV5Hcvrjm3rH6sPfKWUMw85HfYnjn4nH6sgvPsT6KdThsqpZwYTjCEQLGCpyw9Uz4Bmy-bIi4WUvYETgN-TLwGUv3ErHD3njfLrHTvnjnYP1fYPWR4" target="_blank" class="baidu-highlight">ls</a> <a href="https://www.baidu.com/s?wd=-l&tn=44039180_cpr&fenlei=mv6quAkxTZn0IZRqIHckPjm4nH00T1Y4PvD3PHKBujb4njD1P1uW0ZwV5Hcvrjm3rH6sPfKWUMw85HfYnjn4nH6sgvPsT6KdThsqpZwYTjCEQLGCpyw9Uz4Bmy-bIi4WUvYETgN-TLwGUv3ErHD3njfLrHTvnjnYP1fYPWR4" target="_blank" class="baidu-highlight">-l</a> asm-disk*
brw-rw---- 1 grid asmadmin 8, 64 Jul 10 17:31 asm-disk1
brw-rw---- 1 grid asmadmin 8, 208 Jul 10 17:31 asm-disk10
brw-rw---- 1 grid asmadmin 8, 224 Jul 10 17:31 asm-disk11
brw-rw---- 1 grid asmadmin 8, 240 Jul 10 17:31 asm-disk12
brw-rw---- 1 grid asmadmin 8, 80 Jul 10 17:31 asm-disk2
brw-rw---- 1 grid asmadmin 8, 96 Jul 10 17:31 asm-disk3
brw-rw---- 1 grid asmadmin 8, 112 Jul 10 17:31 asm-disk4
brw-rw---- 1 grid asmadmin 8, 128 Jul 10 17:31 asm-disk5
brw-rw---- 1 grid asmadmin 8, 144 Jul 10 17:31 asm-disk6
brw-rw---- 1 grid asmadmin 8, 160 Jul 10 17:31 asm-disk7
brw-rw---- 1 grid asmadmin 8, 176 Jul 10 17:31 asm-disk8
brw-rw---- 1 grid asmadmin 8, 192 Jul 10 17:31 asm-disk9
ASMlib操作系统包安装与配置asm disk磁盘
1.加入6块硬盘，每块100g。不管是热加还是冷加。不管是加硬盘，用san存储划lun，或者再加上多路径，都是能够这么做的。
在操作系统层，能识别这种lun。以下的sdb就是一个刚划分的300g的lun（也就是硬盘。或者裸块设备）。
[[email protected] ~]# cd /dev
[[email protected] dev]# ls -l | grep sd
brw-r----- 1 root disk 8, 0 Apr 11 12:21 sda
brw-r----- 1 root disk 8, 1 Apr 11 12:21 sda1
brw-r----- 1 root disk 8, 2 Apr 11 12:21 sda2
brw-r----- 1 root disk 8, 16 Apr 11 12:21 sdb
分区
我们对硬盘进行分区，兴许把分区分别作为asm磁盘。事实上就一块机械硬盘，根本不是必需分区再做asm。
仅仅有多块硬盘的时候。能够用asm做镜像与条带化等。假设存储已经做了raid，给我划分的lun我也不是必需再做分区，直接能够加为asm了。
当然，比方有/dev/sda。/dev/sdb两块机械硬盘，我能够分别划分出/dev/sda1、/dev/sda2、/dev/sdb1、/dev/sdb2，然后摘取/dev/sda1与/dev/sdb1出来联合做一个asm disk1.
[[email protected] dev]# fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won‘t be recoverable.
The number of cylinders for this disk is set to 39162.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-39162, default 1): 1
Last cylinder or +size or +sizeM or +sizeK (1-39162, default 39162): 60
Command (m for help): n
Command action
e extended
p primary partition (1-4)
2
Invalid partition number for type `2‘
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (61-39162, default 61): 61
Last cylinder or +size or +sizeM or +sizeK (61-39162, default 39162): 120
Command (m for help): n
Command action
e extended
p primary partition (1-4)
3
Invalid partition number for type `3‘
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 3
First cylinder (121-39162, default 121): 121
Last cylinder or +size or +sizeM or +sizeK (121-39162, default 39162): 180
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Selected partition 4
First cylinder (181-39162, default 181): 181
Last cylinder or +size or +sizeM or +sizeK (181-39162, default 39162): 240
Command (m for help): n
You must delete some partition and add an extended partition first
Command (m for help): m
Command action
a toggle a bootable flag
b edit bsd disklabel
c toggle the dos compatibility flag
d delete a partition
l list known partition types
m print this menu
n add a new partition
o create a new empty DOS partition table
p print the partition table
q quit without saving changes
s create a new empty Sun disklabel
t change a partition‘s system id
u change display/entry units
v verify the partition table
w write table to disk and exit
x extra functionality (experts only)
Command (m for help): d
Partition number (1-4): 4
Command (m for help): n
Command action
e extended
p primary partition (1-4)
e
Selected partition 4
First cylinder (181-39162, default 181):
Using default value 181
Last cylinder or +size or +sizeM or +sizeK (181-39162, default 39162): 240
Command (m for help): n
First cylinder (181-240, default 181):
Using default value 181
Last cylinder or +size or +sizeM or +sizeK (181-240, default 240): 210
Command (m for help): n
First cylinder (211-240, default 211):
Using default value 211
Last cylinder or +size or +sizeM or +sizeK (211-240, default 240):
Using default value 240
Command (m for help): m
Command action
a toggle a bootable flag
b edit bsd disklabel
c toggle the dos compatibility flag
d delete a partition
l list known partition types
m print this menu
n add a new partition
o create a new empty DOS partition table
p print the partition table
q quit without saving changes
s create a new empty Sun disklabel
t change a partition‘s system id
u change display/entry units
v verify the partition table
w write table to disk and exit
x extra functionality (experts only)
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[[email protected] dev]# pwd
/dev
[[email protected] dev]# ls | grep sd
sda
sda1
sda2
sdb
sdb1
sdb2
sdb3
sdb4
sdb5
sdb6
我们将sdb这块裸设备，分区称sdb1-sdb6，但这些sdb1-6也并不含文件系统。也就是说，还没被文件系统格式化。
从官方站点下载oracle程序包
pri:orcl2 > uname -a
Linux pri 2.6.18-308.el5 #1 SMP Fri Jan 27 17:17:51 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
oracleasmlib-2.0 - ASM库
oracleasm-support-2.0 - utilities needed to administer ASMLib 管理ASMlib所需的有用程序
oracleasm - a kernel module for the Oracle ASM library
ASM库的内核模块
下载了这5个，一定要注意，看上面5个rpm包，都写着是64位的。
结果最后仅仅能装上这三个。
-rw-r--r-- 1 root root 25460 Apr 11 18:34 oracleasm-2.6.18-308.24.1.el5-2.0.5-1.el5.x86_64.rpm
-rw-r--r-- 1 root root 90225 Apr 11 18:36 oracleasm-support-2.1.8-1.el5.x86_64.rpm
-rw-r--r-- 1 root root 14176 Apr 11 18:36 oracleasmlib-2.0.4-1.el5.x86_64.rpm
对于dji单数据库用户来装，就不用分grid用户与oracle用户了。
据说不建asmadmin等等的组，那么就能够用普通的oracle用户也拥有asmdba权限。
加入每一个节点的/etc/security/limits.conf
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
oracle soft core unlimited
oracle soft core unlimited
加入每一个节点的/etc/pam.d/login文件
session required pam_limits.so
对于Bourne、Bash或Korn shell，将下面内容加到每一个节点的/etc/profile文件里。
if [ $USER = "oracle"] || [ $USER = "grid" ];then
if [ $SHELL = "/bin/ksh" ];then
ulimit -u 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
umask 022
fi
配置ASMLib，不加-i就是显示当前配置，然后运行oracleasm init就等同于之前-i的指令。
/usr/sbin/oracleasm configure -i
配置之前：
[[email protected] asmlib]# /usr/sbin/oracleasm configure
ORACLEASM_ENABLED=false
ORACLEASM_UID=
ORACLEASM_GID=
ORACLEASM_SCANBOOT=true
ORACLEASM_SCANORDER=""
ORACLEASM_SCANEXCLUDE=""
ORACLEASM_USE_LOGICAL_BLOCK_SIZE="false"
配置：
[[email protected] asmlib]# /usr/sbin/oracleasm configure -i
Configuring the Oracle ASM library driver.
This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets (‘[]‘). Hitting <ENTER> without typing an
answer will keep that current value. Ctrl-C will abort.
Default user to own the driver interface []: oracle
Default group to own the driver interface []: dba
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done
配置之后：
[[email protected] asmlib]# /usr/sbin/oracleasm configure
ORACLEASM_ENABLED=true
ORACLEASM_UID=oracle
ORACLEASM_GID=dba
ORACLEASM_SCANBOOT=true
ORACLEASM_SCANORDER=""
ORACLEASM_SCANEXCLUDE=""
ORACLEASM_USE_LOGICAL_BLOCK_SIZE="false"
为asm加入asm磁盘
此时/dev/sdb1仅仅是操作系统认识的分区，我要将其设为让ASM也认识。
/usr/sbin/oracleasm createdisk data1 /dev/sdb1
每一个节点以root输入。ASM磁盘对每一个节点可用。
/usr/sbin/oracleasm scandisks
/usr/sbin/oracleasm listdisks（检验一下）
[[email protected] asmlib]# /usr/sbin/oracleasm createdisk data1 /dev/sdb1
Writing disk header: done
Instantiating disk: failed
Clearing disk header: done
[[email protected] asmlib]#
[[email protected] asmlib]# /usr/sbin/oracleasm createdisk data1 /dev/sdb2
Writing disk header: done
Instantiating disk: failed
Clearing disk header: done
[[email protected] asmlib]# /usr/sbin/oracleasm createdisk data1 /dev/sdb3
Writing disk header: done
Instantiating disk: failed
Clearing disk header: done
[[email protected] asmlib]# /usr/sbin/oracleasm createdisk data1 /dev/sdb4
Writing disk header: failed
Unable to label device "/dev/sdb4"
[[email protected] asmlib]# cd /dev
[[email protected] dev]# ls | grep sd
sda
sda1
sda2
sdb
sdb1
sdb2
sdb3
sdb4
sdb5
sdb6
不知道为什么，我尽管有sdb4,可是这样会失败，是不是扩产分区就不能用来做asm盘的。
每一个节点以root输入，ASM磁盘对每一个节点可用。
[[email protected] dev]# /usr/sbin/oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
以下这个必须配置上，不然不能装GI。这个要跟vip在同一个网段上，而且不能是本机ip，也不能使已经使用的。
/etc/hosts
10.60.17.150 pri-scan
假设出现[ins-40724] no locally defined network interface matches the scan subnet.报错。重新启动下主机就好。
后来又发现有报错，在例如以下这个界面。好像要再再/etc/hosts中，弄一个pri-vip。随便弄一个跟公网不同一个网段，但地址不一样的ip才干通过。
[[email protected] ~]# /usr/sbin/oracleasm createdisk data1 /dev/sdb1
Writing disk header: done
Instantiating disk: failed
Clearing disk header: done
[[email protected] ~]# vi /var/log/oracleasm
Creating /dev/oracleasm mount point: /dev/oracleasm
Loading module "oracleasm": failed
Unable to load module "oracleasm"
Cleaning any stale ASM disks...
Loading module "oracleasm": failed
Unable to load module "oracleasm"
最后发现。官网上给下载的是oracle linux的asm包。
那redhat 的怎么办呢？原来redhat的光碟自带有iso文件。
[[email protected] Server]# ls | grep oracleasm
oracleasm-2.6.18-274.el5-2.0.5-1.el5.x86_64.rpm
oracleasm-2.6.18-274.el5debug-2.0.5-1.el5.x86_64.rpm
oracleasm-2.6.18-274.el5xen-2.0.5-1.el5.x86_64.rpm
oracleasm-support-2.1.7-1.el5.x86_64.rpm
我就是下载了例如以下两个，rpm -ivh 。其余两个安装不了。
oracleasm-support-2.1.7-1.el5.x86_64.rpm
oracleasm-2.6.18-274.el5-2.0.5-1.el5.x86_64.rpm
[[email protected] mnt]# cd /mnt
[[email protected] mnt]# mkdir cdrom
[[email protected] mnt]# mount /dev/cdrom /mnt/cdrom
mount: block device /dev/cdrom is write-protected, mounting read-only
发现还是不行。
将之前下载的64bit的oracleasmlib-2.0.4-1.el5.x86_64.rpm也上传上去，安装上。
也不行。尝试重新启动一下。
[[email protected] ~]# /usr/sbin/oracleasm createdisk data1 /dev/sdc1
Writing disk header: done
Instantiating disk: done
成功了。可能刚才直接重新启动也能够，不用加一个rpm包，也可能要。
反正重新启动是必须的。
[[email protected] ~]# /usr/sbin/oracleasm createdisk data1 /dev/sdc1
Writing disk header: done
Instantiating disk: done
[[email protected] ~]# /usr/sbin/oracleasm createdisk data2 /dev/sdb1
Writing disk header: done
Instantiating disk: done
[[email protected] ~]# /usr/sbin/oracleasm createdisk data3 /dev/sdd1
Writing disk header: done
Instantiating disk: done
[[email protected] ~]# /usr/sbin/oracleasm createdisk data4 /dev/sde1
Writing disk header: done
Instantiating disk: done
[[email protected] ~]# /usr/sbin/oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
[[email protected] ~]#
[[email protected] ~]# /usr/sbin/oracleasm listdisks 或者service oracleasm listdisks
DATA1
DATA2
DATA3
DATA4
有了这4个ASM盘后，開始装GI。
假设要删除 ASM磁盘可通过下面命令：
[[email protected] ~]#/etc/init.d/oracleasm deletedisk ASMDISK5
[[email protected] ~]#/etc/init.d/oracleasm deletedisk ASMDISK5
[[email protected] mapper]# /usr/sbin/oracleasm createdisk data1 /dev/mapper/mpath0p7
Writing disk header: done
Instantiating disk: done
以上是关于安装oracle11grac时使用asmlib和udev的区别的主要内容，如果未能解决你的问题，请参考以下文章
安装 Oracle 11g RAC（空集群节点）时出现问题 INS-35423
如何完全删载与清除oracle 10grac cluster