阿里云ESSD RAC11g搭建

Posted 2022-05-27 qiaomai51

环境规划

物理环境规划

ECS 类型	服务器类型	服务器配置	操作系统版本	Oracle​版本
阿里云	g7se	8C 32G	CentOS 7.9	11.2.0.4.0

磁盘规划

DBNAME	HOSTNAME	ASM 磁盘组	SIZE
orcl	rac1 rac2	OCR	20G
			20G
			20G
		DATA	300G
		ARCH	100G

网络规划

host	public ip	vip	priv ip	scan ip
rac1	172.16.10.1	172.16.10.3	172.16.20.1	172.16.10.5
rac2	172.16.10.2	172.16.10.4	172.16.20.2

阿里云资源创建

创建 VPC

vpc: 172.16.0.0/12

创建交换机

交换机1：172.16.10.0/24

交换机2：172.16.20.0/24

创建 ECS

系统盘：40G

数据盘：100G

自定义密码：root/rootrac_211125

修改主网卡 IP

先停止实例，然后修改。

创建辅助网卡

单独创建弹性网卡作为辅助网卡，可以指定 IP。

创建高可用虚拟IP

专有网络--高可用虚拟IP

配置 keepalived（RAC 可省略）

阿里云 HaVip配合 keepalived 才能实现高可用主备双机，但是 RAC 自带 VIP 的管理功能，可以不用配置 keepalived。

安装 keepalived

yum install keepalived mv /etc/keepalived/keepalived.conf​/etc/keepalived/keepalived.conf.bak

配置

[root@rac1 ~]# vim /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs notification_email [e-mail] [e-mail] [e-mail] notification_email_from [e-mail] smtp_server 192.168.200.1 smtp_connect_timeout 30 router_id LVS_DEVEL vrrp_skip_check_adv_addr vrrp_garp_interval 0 vrrp_gna_interval 0 vrrp_instance VI_1 state MASTER #设置ECS1实例为主实例 interface eth0 #设置网卡名，本示例配置为eth0 virtual_router_id 51 nopreempt priority 100 #设置优先级，数字越大，优先级越高，本示例配置主用实例优先级为100 advert_int 1 authentication auth_type PASS auth_pass 1111 unicast_src_ip 172.16.10.1 #设置ECS实例的私网IP地址 unicast_peer 172.16.10.2 #对端ECS实例的私网IP地址 virtual_ipaddress 172.16.10.3 #设置HaVip的IP地址 172.16.10.5 notify_master "/etc/keepalived/notify_action.sh MASTER" notify_backup "/etc/keepalived/notify_action.sh BACKUP" notify_fault "/etc/keepalived/notify_action.sh FAULT" notify_stop "/etc/keepalived/notify_action.sh STOP" garp_master_delay 1 garp_master_refresh 5 track_interface eth0 #设置ECS实例网卡名，本示例配置为eth0 vrrp_instance VI_2 state BACKUP #设置ECS1实例为备用实例 interface eth0 #设置网卡名，本示例配置为eth0 virtual_router_id 52 nopreempt priority 10 #设置优先级，数字越大，优先级越高，本示例配置备用实例优先级为10 advert_int 1 authentication auth_type PASS auth_pass 1111 unicast_src_ip 172.16.10.1 #设置ECS实例的私网IP地址 unicast_peer 172.16.10.2 #对端ECS实例的私网IP地址 virtual_ipaddress 172.16.10.4 #设置HaVip的IP地址 notify_master "/etc/keepalived/notify_action.sh MASTER" notify_backup "/etc/keepalived/notify_action.sh BACKUP" notify_fault "/etc/keepalived/notify_action.sh FAULT" notify_stop "/etc/keepalived/notify_action.sh STOP" garp_master_delay 1 garp_master_refresh 5 track_interface eth0 #设置ECS实例网卡名，本示例配置为eth0 service keepalived start # 启动服务 [root@rac2 ~]# vim /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs notification_email [e-mail] [e-mail] [e-mail] notification_email_from [e-mail] smtp_server 192.168.200.1 smtp_connect_timeout 30 router_id LVS_DEVEL vrrp_skip_check_adv_addr vrrp_garp_interval 0 vrrp_gna_interval 0 vrrp_instance VI_1 state BACKUP #设置ECS2实例为备用实例 interface eth0 #设置网卡名，本示例配置为eth0 virtual_router_id 51 nopreempt priority 10 #设置优先级，数字越大，优先级越高，本示例配置备用实例优先级为10 advert_int 1 authentication auth_type PASS auth_pass 1111 unicast_src_ip 172.16.10.2 #设置ECS实例的私网IP地址 unicast_peer 172.16.10.1 #对端ECS实例的私网IP地址 virtual_ipaddress 172.16.10.3 #设置HaVip的IP地址 172.16.10.5 notify_master "/etc/keepalived/notify_action.sh MASTER" notify_backup "/etc/keepalived/notify_action.sh BACKUP" notify_fault "/etc/keepalived/notify_action.sh FAULT" notify_stop "/etc/keepalived/notify_action.sh STOP" garp_master_delay 1 garp_master_refresh 5 track_interface eth0 #设置ECS实例网卡名，本示例配置为eth0 vrrp_instance VI_2 state MASTER #设置ECS1实例为主实例 interface eth0 #设置网卡名，本示例配置为eth0 virtual_router_id 52 nopreempt priority 100 #设置优先级，数字越大，优先级越高，本示例配置主用实例优先级为100 advert_int 1 authentication auth_type PASS auth_pass 1111 unicast_src_ip 172.16.10.2 #设置ECS实例的私网IP地址 unicast_peer 172.16.10.1 #对端ECS实例的私网IP地址 virtual_ipaddress 172.16.10.4 #设置HaVip的IP地址 notify_master "/etc/keepalived/notify_action.sh MASTER" notify_backup "/etc/keepalived/notify_action.sh BACKUP" notify_fault "/etc/keepalived/notify_action.sh FAULT" notify_stop "/etc/keepalived/notify_action.sh STOP" garp_master_delay 1 garp_master_refresh 5 track_interface eth0 #设置ECS实例网卡名，本示例配置为eth0 service keepalived start # 启动服务

创建共享磁盘并挂载

环境配置

配置网卡 eth1

ip add vi /etc/sysconfig/network-scripts/ifcfg-eth1 DEVICE=eth1 BOOTPROTO=dhcp ONBOOT=yes TYPE=Ethernet USERCTL=yes PEERDNS=no IPV6INIT=no PERSISTENT_DHCLIENT=yes HWADDR=00:16:3e:17:**:** # ip add 获取的 eth1 的 mac 地址 DEFROUTE=no # 表示网卡接口不是默认路由。为避免在启动（ifup）辅助弹性网卡时改变 ECS 实例活动的默认路由，不要将 eth1 设置为默认路由 service network restart

修改主机名

节点1: hostname rac1 vi /etc/hostname rac1 节点2: hostname rac2 vi /etc/hostname rac2

关闭防火墙、selinux

阿里云默认已关闭

service firewalld status getenforce

关闭系统自带的时钟同步服务

systemctl stop chronyd.service systemctl disable chronyd.service systemctl status chronyd.service mv /etc/chrony.conf /etc/chrony.conf.bak mv /etc/ntp.conf /etc/ntp.conf.bak --即使 ntpd 服务不存在，但配置文件依然要删除，否则 RAC 会报错 CRS-2409

关闭透明大页

cat /sys/kernel/mm/transparent_hugepage/enabled [always] madvise never --临时关闭，重启后会失效 echo never > /sys/kernel/mm/transparent_hugepage/enabled cat /sys/kernel/mm/transparent_hugepage/enabled always madvise [never] --永久关闭 chmod 744 /etc/rc.d/rc.local vi /etc/rc.d/rc.local echo never > /sys/kernel/mm/transparent_hugepage/enabled

配置 /etc/hosts 文件

阿里云服务器删除实例名那一行，否则节点识别会出现问题

cat <<EOF >>/etc/hosts # public ip 172.16.10.1 rac1 172.16.10.2 rac2 # vip 172.16.10.3 rac1-vip 172.16.10.4 rac2-vip # priv ip 172.16.20.1 rac1-priv 172.16.20.2 rac2-priv # scan ip 172.16.10.5 scan EOF

配置内核参数

kernel.sched_rt_runtime_us=-1 解决 Centos 7.9 集群 cssd 服务无法启动问题。

cat <<EOF >>/etc/sysctl.conf fs.aio-max-nr = 1048576 fs.file-max = 6815744 kernel.shmall = 32*0.9*1024*1024*1024/4096 kernel.shmmax = 32*0.9*1024*1024*1024 kernel.shmmni = 4096 kernel.sem = 250 32000 100 128 net.ipv4.ip_local_port_range = 9000 65500 net.core.rmem_default = 262144 net.core.rmem_max = 4194304 net.core.wmem_default = 262144 net.core.wmem_max = 1048576 kernel.sched_rt_runtime_us=-1 EOF /sbin/sysctl -p # 使更改生效

为 oracle 和 grid 用户设置 shell 限制

cat <<EOF >>/etc/security/limits.conf oracle soft nproc 2047 oracle hard nproc 16384 oracle soft nofile 1024 oracle hard nofile 65536 grid soft nproc 2047 grid hard nproc 16384 grid soft nofile 1024 grid hard nofile 65536 EOF

配置 swap

swap 推荐设置如下： RAM Swap Space 1-2G 1.5times 2-16G equal >16G 16G dd if=/dev/zero of=/swapfile bs=8192 count=2M mkswap /swapfile # 格式化 swap chown root:root /swapfile chmod 0600 /swapfile swapon /swapfile # 激活 swap swapon -s # 查看是否生效 重启时自动挂载 swap vi /etc/fstab /swapfile swap swap defaults 0 0

挂载数据盘

su - root mkfs -t ext4 /dev/nvme1n1 mkdir /u01 vi /etc/fstab /dev/nvme1n1 /u01 ext4 defaults 0 0 mount -a

新建用户、组

groupadd oinstall groupadd dba groupadd oper groupadd asmadmin groupadd asmdba groupadd asmoper useradd -g oinstall -G dba,oper,asmadmin,asmdba,asmoper -s /bin/bash -d /home/grid grid useradd -g oinstall -G dba,oper,asmadmin,asmdba,asmoper -s /bin/bash -d /home/oracle oracle mkdir -p /u01/app/grid mkdir -p /u01/app/11.2.0/grid mkdir -p /u01/app/oracle mkdir -p /u01/soft chown -R grid:oinstall /u01 chown -R oracle:oinstall /u01/app/oracle chmod -R 775 /u01 passwd grid passwd oracle

配置用户环境变量

配置 grid 用户环境变量

节点2 ORACLE_SID=+ASM2 [grid@rac1 ~]$ cat <<EOF >>.bash_profile umask 022 export ORACLE_SID=+ASM1 export ORACLE_BASE=/u01/app/grid export ORACLE_HOME=/u01/app/11.2.0/grid export PATH=\\$PATH:\\$ORACLE_HOME/bin alias sqlplus=rlwrap sqlplus EOF

配置 oracle 用户环境变量

节点2 ORACLE_SID=orcl2 [oracle@rac1 ~]$ cat <<EOF >>.bash_profile umask 022 export ORACLE_SID=orcl1 export ORACLE_BASE=/u01/app/oracle export ORACLE_HOME=\\$ORACLE_BASE/product/11.2.0/dbhome_1 export PATH=\\$PATH:\\$ORACLE_HOME/bin export LD_LIBRARY_PATH=/lib:/usr/lib:/usr/local/lib:\\$ORACLE_HOME/lib alias sqlplus=rlwrap sqlplus EOF

配置 SSH 用户等效性

grid 用户下配置 ssh 等效性

#​创建密钥（2个节点） ssh-keygen -t rsa cd ~/.ssh cat id_rsa.pub >>authorized_keys #​将 rac2 的密钥添加到 rac1 [grid@rac1 ~]$ ssh grid@rac2 cat /home/grid/.ssh/authorized_keys >>/home/grid/.ssh/authorized_keys # 将 rac1 的密钥添加到 rac2 ESSD技术解读-01 云原生时代，阿里云 ESSD 快照服务 助力企业级数据保护 ESSD技术解读-01 云原生时代，阿里云 ESSD 快照服务 助力企业级数据保护 阿里云数据库存储类型ESSD云盘/本地SSD盘/SSD云盘/区别选择攻略！ 深圳云栖大会阿里云弹性计算ESSD云盘产品全面解析 ESSD技术解读 云原生时代，阿里云块存储 ESSD 快照服务如何被企业级数据保护所集成？ 阿里云几种云盘类型如何选择有啥区别