之前写过一篇双网卡绑定的文章，如下：
http://blog.csdn.net/tianlesoftware/article/details/6189639
http://blog.csdn.net/tianlesoftware/article/details/5362925
这篇看一个完整的RAC修改网卡绑定的例子。
一．原RAC信息
数据库：11.2.0.3
操作系统: Redhat 5.8
[[email protected] ~]# cat /etc/hosts
127.0.0.1 localhost
192.168.16.200 rac1 rac1-public
192.168.0.100 rac1-priv
192.168.16.201 rac1-vip
192.168.16.202 rac2 rac2-public
192.168.0.200 rac2-priv
192.168.16.203 rac2-vip
192.168.16.207 rac-scan
每个节点的服务器有4个网卡。 eth0对应Public IP，eth1 对应Private IP。
[[email protected] ~]# srvctl config nodeapps
Network exists:1/192.168.16.0/255.255.255.0/eth0, type static
VIP exists:/rac1-vip/192.168.16.201/192.168.16.0/255.255.255.0/eth0, hosting node rac1
VIP exists:/rac2-vip/192.168.16.203/192.168.16.0/255.255.255.0/eth0, hosting node rac2
GSD exists
ONS exists: Local port 6100, remote port6200, EM port 2016
[[email protected] ~]# srvctl config nodeapps -nrac1 -a
-n <node_name> option has beendeprecated.
Network exists:1/192.168.16.0/255.255.255.0/eth0, type static
VIP exists: /rac1-vip/192.168.16.201/192.168.16.0/255.255.255.0/eth0,hosting node rac1
[[email protected] ~]# srvctl config nodeapps -nrac2 -a
-n <node_name> option has beendeprecated.
Network exists:1/192.168.16.0/255.255.255.0/eth0, type static
VIP exists: /rac2-vip/192.168.16.203/192.168.16.0/255.255.255.0/eth0,hosting node rac2
[[email protected] ~]#
二．修改RAC 配置
之前的测试是先修改双网卡的绑定，但是一旦修改之后，RAC就不能启动，也就不能进行修改，所以我们这里先修改。修改完之后在该双网卡。
2.1 备份OCR和VOTINGDISK
[[email protected] ~]# crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1.ONLINE 6be9df5e116e4fa6bfea0a06cec3a0e5(/dev/asm-diskb) [OCRVOTING]
2.ONLINE 0935e75b0c3e4fe3bf34ec484d7a332e(/dev/asm-diskc) [OCRVOTING]
3.ONLINE 9191596fff034f6abfcdaf3b77dbadd9(/dev/asm-diskd) [OCRVOTING]
4.ONLINE 3ed8af5882454ffdbf1ac89b47531b96(/dev/asm-diske) [OCRVOTING]
5.ONLINE 52cb435078424fc5bfd693bdb0e6771b(/dev/asm-diskf) [OCRVOTING]
[[email protected] ~]# dd if=/dev/asm-diskbof=/u01/backup/voting_disk.bak
2097152+0 records in
2097152+0 records out
1073741824 bytes (1.1 GB) copied, 15.5894seconds, 68.9 MB/s
[[email protected] ~]# ocrconfig -export/u01/backup/ocr.exp
[[email protected] ~]# ocrconfig -showbackup
rac2 2013/02/25 20:28:57 /u01/app/grid/11.2.0/cdata/rac-cluster/backup00.ocr
rac2 2013/02/25 16:28:57 /u01/app/grid/11.2.0/cdata/rac-cluster/backup01.ocr
rac2 2013/02/25 12:28:57 /u01/app/grid/11.2.0/cdata/rac-cluster/backup02.ocr
rac1 2013/02/24 06:05:17 /u01/app/grid/11.2.0/cdata/rac-cluster/day.ocr
rac1 2013/02/19 18:18:35 /u01/app/grid/11.2.0/cdata/rac-cluster/week.ocr
2.1 修改网卡和IP信息
2.1.1 RAC 进程信息
[[email protected] ~]# crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.DATA.dg ora....up.type ONLINE ONLINE rac1
ora.FRA.dg ora....up.type ONLINE ONLINE rac1
ora....ER.lsnr ora....er.type ONLINE ONLINE rac1
ora....N1.lsnr ora....er.type ONLINE ONLINE rac1
ora....TING.dg ora....up.type ONLINE ONLINE rac1
ora.asm ora.asm.type ONLINE ONLINE rac1
ora.cd.db ora....se.type ONLINE ONLINE rac1
ora.cvu ora.cvu.type ONLINE ONLINE rac1
ora.gsd ora.gsd.type OFFLINE OFFLINE
ora....network ora....rk.type ONLINE ONLINE rac1
ora.oc4j ora.oc4j.type ONLINE ONLINE rac1
ora.ons ora.ons.type ONLINE ONLINE rac1
ora....SM1.asm application ONLINE ONLINE rac1
ora....C1.lsnr application ONLINE ONLINE rac1
ora.rac1.gsd application OFFLINE OFFLINE
ora.rac1.ons application ONLINE ONLINE rac1
ora.rac1.vip ora....t1.type ONLINE ONLINE rac1
ora....SM2.asm application ONLINE ONLINE rac2
ora....C2.lsnr application ONLINE ONLINE rac2
ora.rac2.gsd application OFFLINE OFFLINE
ora.rac2.ons application ONLINE ONLINE rac2
ora.rac2.vip ora....t1.type ONLINE ONLINE rac2
ora....ry.acfs ora....fs.type ONLINE OFFLINE
ora.scan1.vip ora....ip.type ONLINE ONLINE rac1
2.1.2 查看当前配置
[[email protected] ~]# oifcfg getif -global
eth1 192.168.0.0 global cluster_interconnect
eth0 192.168.16.0 global public
2.1.3 修改PUBLIC和 Private信息
--删除当前配置
[[email protected] ~]# oifcfg delif -global eth0
[[email protected] ~]# oifcfg delif -global eth1
PRIF-31: Failed to deletethe specified network interface because it is the last private interface
--这里我们要先添加才能删除。
[[email protected] ~]# oifcfg getif
eth1 192.168.0.0 global cluster_interconnect
[[email protected] ~]# oifcfg setif -globalbond0/192.168.16.0:public
[[email protected] ~]# oifcfg setif -globalbond1/192.168.0.0:cluster_interconnect
[[email protected] ~]# oifcfg getif
eth1 192.168.0.0 global cluster_interconnect
bond0 192.168.16.0 global public
bond1 192.168.0.0 global cluster_interconnect
[[email protected] ~]# oifcfg delif -global eth1
[[email protected] ~]# oifcfg getif
bond0 192.168.16.0 global public
bond1 192.168.0.0 global cluster_interconnect
2.1.4 修改VIP
[[email protected] ~]# srvctl modify nodeapps -nrac1 -A 192.168.16.201/255.255.255.0/bond0
PRKO-2200 : Invalid interface specified oncommand line: bond0
--提示bond0没有激活，先把bond0的配置信息写好，激活，先改我们CRS的信息：
[[email protected] network-scripts]# ifup bond0
--再次修改就ok：
[[email protected] ~]# srvctl modify nodeapps -nrac1 -A 192.168.16.201/255.255.255.0/bond0
[[email protected] ~]# srvctl modify nodeapps -nrac2 -A 192.168.16.203/255.255.255.0/bond0
[[email protected] bin]# ./oifcfg delif -globaleth0
[[email protected] bin]# ./oifcfgdelif -global eth1
[[email protected] bin]# ./oifcfg getif
2.1.5 检查监听等配置中的IP 信息
因为我这里只是修改了一下网卡的名称，所以不需要检查，IP地址是一样的。
三．双网卡绑定配置
3.1 停止RAC
# crsctl stop crs
3.1 节点1操作
创建/etc/sysconfig/network-scripts/ifcfg-bond0文件，内容如下：
BOOTPROTO=none
BROADCAST=192.168.16.255
IPADDR=192.168.16.200
NETMASK=255.255.255.0
ONBOOT=yes
TYPE=Ethernet
GATEWAY=192.168.16.254
USERCTL=no
IPV6INIT=no
PEERDNS=yes
创建/etc/sysconfig/network-scripts/ifcfg-bond1文件，内容如下：
BOOTPROTO=none
BROADCAST=192.168.0.255
IPADDR=192.168.0.100
NETMASK=255.255.255.0
ONBOOT=yes
TYPE=Ethernet
GATEWAY=192.168.0.254
USERCTL=no
IPV6INIT=no
PEERDNS=yes
分别修改原来网卡配置文件的信息，删除其中的IP 地址、子网掩码等信息
[[email protected] network-scripts]# cat ifcfg-eth0
# Broadcom Corporation NetXtreme BCM5720Gigabit Ethernet PCIe
DEVICE=eth0
BOOTPROTO=none
ONBOOT=yes
TYPE=Ethernet
[[email protected] network-scripts]# cat ifcfg-eth1
# Broadcom Corporation NetXtreme BCM5720Gigabit Ethernet PCIe
DEVICE=eth1
BOOTPROTO=none
ONBOOT=yes
TYPE=Ethernet
[[email protected] network-scripts]# cat ifcfg-eth2
# Broadcom Corporation NetXtreme BCM5720Gigabit Ethernet PCIe
DEVICE=eth2
BOOTPROTO=none
ONBOOT=yes
TYPE=Ethernet
[[email protected] network-scripts]# cat ifcfg-eth3
# Broadcom Corporation NetXtreme BCM5720Gigabit Ethernet PCIe
DEVICE=eth3
BOOTPROTO=none
ONBOOT=yes
TYPE=Ethernet
[[email protected] network-scripts]#
在/etc/modprobe.conf文件中添加如下内容：
alias bond1 bonding
options bond0 miimon=100 mode=0
options bond1 miimon=100 mode=0
说明：
（1）、miimon 是链路监测的时间间隔单位是毫秒，miimon=100的意思就是，每100毫秒检测网卡和交换机之间是否连通，如不通则使用另外的链路。
（2)、mode=0 表示负载均衡方式，两块网卡都工作，需要交换机作支持
mode=1 表示冗余方式，网卡只有一个工作，一个出问题启用另外的
mode=6 表示负载均衡方式，两块网卡都工作，不需要交换机作支持
2.2 节点2操作
创建/etc/sysconfig/network-scripts/ifcfg-bond0文件，内容如下：
DEVICE=bond0
BOOTPROTO=none
BROADCAST=192.168.16.255
IPADDR=192.168.16.202
NETMASK=255.255.255.0
ONBOOT=yes
TYPE=Ethernet
GATEWAY=192.168.16.254
USERCTL=no
IPV6INIT=no
PEERDNS=yes
创建/etc/sysconfig/network-scripts/ifcfg-bond1文件，内容如下：
DEVICE=bond1
BOOTPROTO=none
BROADCAST=192.168.0.255
IPADDR=192.168.0.200
NETMASK=255.255.255.0
ONBOOT=yes
TYPE=Ethernet
GATEWAY=192.168.0.254
USERCTL=no
IPV6INIT=no
PEERDNS=yes
分别修改原来网卡配置文件的信息，删除其中的IP 地址、子网掩码等信息
[[email protected] network-scripts]# cat ifcfg-eth0
# Broadcom Corporation NetXtreme BCM5720Gigabit Ethernet PCIe
DEVICE=eth0
BOOTPROTO=none
ONBOOT=yes
TYPE=Ethernet
[[email protected] network-scripts]# cat ifcfg-eth1
# Broadcom Corporation NetXtreme BCM5720Gigabit Ethernet PCIe
DEVICE=eth1
BOOTPROTO=none
ONBOOT=yes
TYPE=Ethernet
[[email protected] network-scripts]# cat ifcfg-eth2
# Broadcom Corporation NetXtreme BCM5720Gigabit Ethernet PCIe
DEVICE=eth2
BOOTPROTO=none
ONBOOT=yes
TYPE=Ethernet
[[email protected] network-scripts]# cat ifcfg-eth3
# Broadcom Corporation NetXtreme BCM5720Gigabit Ethernet PCIe
DEVICE=eth3
BOOTPROTO=none
ONBOOT=yes
TYPE=Ethernet
[[email protected] network-scripts]#
alias bond0bonding
alias bond1 bonding
options bond0 miimon=100 mode=0
options bond1 miimon=100 mode=0
说明：
（1）、miimon 是链路监测的时间间隔单位是毫秒，miimon=100的意思就是，每100毫秒检测网卡和交换机之间是否连通，如不通则使用另外的链路。
（2)、mode=0 表示负载均衡方式，两块网卡都工作，需要交换机作支持
mode=1 表示冗余方式，网卡只有一个工作，一个出问题启用另外的
mode=6 表示负载均衡方式，两块网卡都工作，不需要交换机作支持
2.3. 执行绑定
--节点1执行绑定：
[[email protected] network-scripts]# ifenslavebond0 eth1 eth0
Illegal operation; the specified masterinterface ‘bond0‘ is not up.
[[email protected] network-scripts]# ifup bond0
[[email protected] network-scripts]# ifenslavebond0 eth1 eth0
[root[email protected] network-scripts]#
[[email protected] network-scripts]# ifup bond1
[[email protected] network-scripts]# ifenslavebond1 eth2 eth3
--节点2执行绑定：
[[email protected] network-scripts]# ifup bond0
[[email protected] network-scripts]# ifup bond1
[[email protected] network-scripts]# ifenslavebond0 eth1 eth0
[[email protected] network-scripts]# ifenslavebond1 eth2 eth3
[[email protected] network-scripts]#
2.4 设置开机自动绑定
在2个节点分别将如下参数添加到/etc/rc.local中，让开机自动绑定：
ifenslave bond0 eth0 eth1
ifenslave bond1 eth2 eth3
2.5 重新网络
[[email protected] network-scripts]# servicenetwork restart
[[email protected] network-scripts]# servicenetwork restart
绑定网卡的启动与关闭：
# ifdown bond0
#ifup bond0
2.6 查看网卡信息
2.6.1 节点1
[[email protected] /]# ifconfig
bond0 Link encap:Ethernet HWaddr90:B1:1C:31:F0:E2
inet addr:192.168.16.200 Bcast:192.168.16.255 Mask:255.255.255.0
inet6 addr: fe80::92b1:1cff:fe31:f0e2/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:116 errors:0 dropped:0 overruns:0 frame:0
TX packets:41 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:10178 (9.9 KiB) TX bytes:12096 (11.8 KiB)
bond1 Link encap:Ethernet HWaddr90:B1:1C:31:F0:E3
inet addr:192.168.0.100 Bcast:192.168.0.255 Mask:255.255.255.0
inet6 addr: fe80::92b1:1cff:fe31:f0e3/64 Scope:Link
UP BROADCAST RUNNING MASTERMULTICAST MTU:1500 Metric:1
RX packets:53 errors:0 dropped:0 overruns:0 frame:0
TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:4214 (4.1 KiB) TXbytes:334 (334.0 b)
eth0 Link encap:Ethernet HWaddr90:B1:1C:31:F0:E2
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:59 errors:0 dropped:0 overruns:0 frame:0
TX packets:21 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:5530 (5.4 KiB) TXbytes:5134 (5.0 KiB)
Interrupt:162 Memory:d51a0000-d51b0000
eth1 Link encap:Ethernet HWaddr90:B1:1C:31:F0:E2
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:57 errors:0 dropped:0 overruns:0 frame:0
TX packets:20 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:4648 (4.5 KiB) TXbytes:6962 (6.7 KiB)
Interrupt:170 Memory:d51d0000-d51e0000
eth2 Link encap:Ethernet HWaddr90:B1:1C:31:F0:E3
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:36 errors:0 dropped:0 overruns:0 frame:0
TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2715 (2.6 KiB) TXbytes:334 (334.0 b)
Interrupt:178 Memory:d50a0000-d50b0000
eth3 Link encap:Ethernet HWaddr90:B1:1C:31:F0:E3
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:17 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1499 (1.4 KiB) TXbytes:0 (0.0 b)
Interrupt:186 Memory:d50d0000-d50e0000
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:813162 errors:0 dropped:0 overruns:0 frame:0
TX packets:813162 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:406199603 (387.3 MiB) TXbytes:406199603 (387.3 MiB)
virbr0 Link encap:Ethernet HWaddr00:00:00:00:00:00
inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:144 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:27422(26.7 KiB)
[[email protected] /]#
2.6.2 节点2
[[email protected] /]# ifconfig
bond0 Link encap:Ethernet HWaddr90:B1:1C:30:5A:7C
inet addr:192.168.16.202 Bcast:192.168.16.255 Mask:255.255.255.0
inet6 addr: fe80::92b1:1cff:fe30:5a7c/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:98 errors:0 dropped:0 overruns:0 frame:0
TX packets:30 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:8096 (7.9 KiB) TXbytes:9710 (9.4 KiB)
bond1 Link encap:Ethernet HWaddr 90:B1:1C:30:5A:7D
inet addr:192.168.0.200 Bcast:192.168.0.255 Mask:255.255.255.0
inet6 addr: fe80::92b1:1cff:fe30:5a7d/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:87 errors:0 dropped:0 overruns:0 frame:0
TX packets:1 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:6820 (6.6 KiB) TXbytes:64 (64.0 b)
eth0 Link encap:Ethernet HWaddr90:B1:1C:30:5A:7C
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:38 errors:0 dropped:0 overruns:0 frame:0
TX packets:11 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3302 (3.2 KiB) TXbytes:1374 (1.3 KiB)
Interrupt:162 Memory:d51a0000-d51b0000
eth1 Link encap:Ethernet HWaddr90:B1:1C:30:5A:7C
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:60 errors:0 dropped:0 overruns:0 frame:0
TX packets:19 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:4794 (4.6 KiB) TXbytes:8336 (8.1 KiB)
Interrupt:170 Memory:d51d0000-d51e0000
eth2 Link encap:Ethernet HWaddr90:B1:1C:30:5A:7D
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:44 errors:0 dropped:0 overruns:0 frame:0
TX packets:1 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3442 (3.3 KiB) TXbytes:64 (64.0 b)
Interrupt:178 Memory:d50a0000-d50b0000
eth3 Link encap:Ethernet HWaddr90:B1:1C:30:5A:7D
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:43 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3378 (3.2 KiB) TXbytes:0 (0.0 b)
Interrupt:186 Memory:d50d0000-d50e0000
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:1108104 errors:0 dropped:0 overruns:0 frame:0
TX packets:1108104 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:387043309 (369.1 MiB) TXbytes:387043309 (369.1 MiB)
virbr0 Link encap:Ethernet HWaddr00:00:00:00:00:00
inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:140 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:24322(23.7 KiB)
[[email protected] /]#
2.7 启动RAC 验证
[[email protected] /]# crsctl start crs
CRS-4123: Oracle High Availability Serviceshas been started.
[[email protected] ~]# crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.DATA.dg ora....up.type ONLINE ONLINE rac1
ora.FRA.dg ora....up.type ONLINE ONLINE rac1
ora....ER.lsnr ora....er.type ONLINE ONLINE rac1
ora....N1.lsnr ora....er.type ONLINE ONLINE rac1
ora....TING.dg ora....up.type ONLINE ONLINE rac1
ora.asm ora.asm.type ONLINE ONLINE rac1
ora.cd.db ora....se.type OFFLINE OFFLINE
ora.cvu ora.cvu.type ONLINE ONLINE rac1
ora.gsd ora.gsd.type OFFLINE OFFLINE
ora....network ora....rk.type ONLINE ONLINE rac1
ora.oc4j ora.oc4j.type ONLINE ONLINE rac1
ora.ons ora.ons.type ONLINE ONLINE rac1
ora....SM1.asm application ONLINE ONLINE rac1
ora....C1.lsnr application ONLINE ONLINE rac1
ora.rac1.gsd application OFFLINE OFFLINE
ora.rac1.ons application ONLINE ONLINE rac1
ora.rac1.vip ora....t1.type ONLINE ONLINE rac1
ora....SM2.asm application ONLINE ONLINE rac2
ora....C2.lsnr application ONLINE ONLINE rac2
ora.rac2.gsd application OFFLINE OFFLINE
ora.rac2.ons application ONLINE ONLINE rac2
ora.rac2.vip ora....t1.type ONLINE ONLINE rac2
ora....ry.acfs ora....fs.type ONLINE OFFLINE
ora.scan1.vip ora....ip.type ONLINE ONLINE rac1
MOS 文档：
How to Modify Private Network Informationin Oracle Clusterware [ID 283684.1]