Oracle11gR2 RAC Install On CentOS6.5小结

前言

本小结借鉴于网上各种Oracle RAC安装说明，只记录大致操作，对于安装过程中遇到的问题及具体安装细节未做过多说明，若要参考此文档，需根据自身具体环境而定！

环境准备

硬件

  • 节点服务器*2
项目 说明
CPU Intel(R) Xeon(R) CPU E5506 @ 2.13GHz *8
内存 64G
硬盘 4*500G 7500转以上
HBA 2块（网络冗余）
网卡 3个千兆网口
  • SAN共享存储服务器*1
项目 说明
缓存 32G
容量 12T 可扩展
端口 4个FC

软件

1
2
3
OS:CentOS 6.5 x86_64
数据库软件：Oracle 11.2.0.4
集群套件：Oracle Grid 11.2.0.4

架构及规划

拓扑

配置规划

  • ** 节点配置 **
节点名称 实例名称 数据库名称 处理器 RAM 操作系统
rac1 wlc1 wwlc 8 32GB CentOS 6.5 x86_64
rac2 wlc2 wwlc 8 32GB CentOS 6.5 x86_64
  • ** 网络配置 **
节点名称 Public IP Private IP VIP SCAN名称 SCAN IP
rac1 192.168.199.11 192.168.1.11 192.168.199.13 scan 192.168.199.15
rac2 192.168.199.12 192.168.1.12 192.168.199.14 scan 192.168.199.15
  • ** 软件组件 **
软件组件 操作系统用户 主组 辅助组 Oracle基目录 Oracle主目录
Grid infra grid oinstall asmadmin、asmdba、asmoper /u01/app/grid_base /u01/app/grid_home
Oracle RAC oracle oinstall dba、oper、asmdba /u01/app/oracle /u01/app/oracle/p roduct/11.2.0/db
  • ** 存储组件 **
存储组件 类型 卷大小 ASM卷组名 ASM冗余 设备名
OCR/VF ASM 6G OCR External /dev/asm_diskb
数据 ASM 1.5T DATA External /dev/asm_diskc
归档 ASM 1.5T ARCH External /dev/asm_diskd

系统准备

修改主机名

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
$vi /etc/sysconfig/network
节点1
NETWORKING=yes
HOSTNAME=rac1
NETWORKING_IPV6=no
PEERNTP=no
GATEWAY=192.168.199.1
NOZEROCONF=yes
节点2
NETWORKING=yes
HOSTNAME=rac2
NETWORKING_IPV6=no
PEERNTP=no
GATEWAY=192.168.199.1
NOZEROCONF=yes
NOZEROCONF=ye

修改/etc/hosts

1
2
3
4
5
6
7
8
节点1、节点2
192.168.199.11 rac1
192.168.199.12 rac2
192.168.199.13 rac1-vip
192.168.199.14 rac2-vip
192.168.1.11 rac1-priv
192.168.1.12 rac2-priv
192.168.199.15 scan-ip

修改内核参数

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
$vi /etc/sysctl.conf
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall =7864320 #内存大小/4096分页大小 
kernel.shmmax = 52451655680 #内存大小
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586
vm.nr_hugepages = sga/pagesize  #开启hugepage，oracle锁定内存防止换出

$sysctl –p

配置NTP

oracle rac必须进行时钟同步，如果没有时钟同步，按照下面的方式配置，让oracle自身来解决

1
2
3
$/sbin/service ntpd stop
$chkconfig ntpd off
$mv /etc/ntp.conf /etc/ntp.conf.org

关闭防火墙

1
2
3
4
5
6
7
8
$chkconfig --list iptables
$chkconfig iptables off
$chkconfig --list iptables
$service iptables stop
$service network restart
关闭selinux
$vi /etc/selinux/config
disable

修改用户限制

1
2
3
4
5
6
7
8
9
10
11
12
13
14
$vi /etc/security/limits.conf
oracle          soft   nofile         4096
oracle          hard   nofile         65536
oracle          soft   nproc          2047
oracle          hard   nproc          16384
oracle          soft   stack          10240
grid            soft   nofile         4096
grid            hard   nofile         65536
grid            soft   nproc          2047
grid            hard   nproc          16384
grid            soft   stack          10240
*               soft   memlock        18874368
*               hard   memlock        18874368
#以上memlock<ram的大小 单位为k

vi /etc/pam.d/login

1
echo "session    required     pam_limits.so" >>/etc/pam.d/login

安装OS软件包

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
$rpm -q binutils \
compat-libstdc++-33 \
elfutils-libelf \
elfutils-libelf-devel \
gcc \
gcc-c++ \
glibc \
glibc-common \
glibc-devel \
glibc-headers \
ksh \
libaio \
libaio-devel \
libgcc \
libstdc++ \
libstdc++-devel \
libXp \
make \
numactl-devel \
sysstat \
unixODBC \
unixODBC-devel \
compat-libcap1.x86_64 \
libcap.x86_64|grep not
一般要安装下面的软件包
$yum install -y compat-libstdc++-33
$yum install -y elfutils-libelf-devel
$yum install -y gcc-c++
$yum install -y ksh
$yum install -y libaio-devel
$yum install -y libstdc++-devel
$yum install -y libXp
$yum install -y numactl-devel
$yum install -y unixODBC
$yum install -y unixODBC-devel
$yum install -y compat-libcap1.x86_64

格式化共享磁盘

存储直接划分3个存储组件所需磁盘
Disk /dev/sdb: 6GB
Disk /dev/sdc: 1.5T
Disk /dev/sdd: 1.5T

UDEV绑定磁盘

1
2
3
4
$for i in b c d ;
do
echo "KERNEL==\"sd*\", BUS==\"scsi\", PROGRAM==\"/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/\$name\", RESULT==\"`/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/sd$i`\", NAME=\"asm-disk$i\", OWNER=\"grid\", GROUP=\"asmadmin\", MODE=\"0660\""  >> /etc/udev/rules.d/99-oracle-asmdevices.rules 
done

创建grid、oracle用户

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
$/usr/sbin/groupadd -g 501 oinstall
$/usr/sbin/groupadd -g 502 asmadmin
$/usr/sbin/groupadd -g 503 dba
$/usr/sbin/groupadd -g 504 oper
$/usr/sbin/groupadd -g 505 asmdba
$/usr/sbin/groupadd -g 506 asmoper
$/usr/sbin/useradd -u 501 -g oinstall -G asmadmin,asmdba,asmoper,oper,dba grid
$/usr/sbin/useradd -u 502 -g oinstall -G dba,asmdba,oper oracle

#配置环境变量
$su- grid
$Vi .bash_profile
umask 022
export ORACLE_BASE=/u01/app/grid_base
export ORACLE_HOME=/u01/app/grid_home
export ORACLE_SID=+ASM1  #第二个节点+ASM2
export PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$PATH

$su- oracle
$Vi .bash_profile
umask 022
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1
export ORA_GRID_HOME=/u01/app/grid_home
export ORACLE_UNQNAME=wwlc  
export ORACLE_SID=wwlc1 #第二个节点wwlc2  
export PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ORACLE_HOME/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
export NLS_LANG=AMERICAN_AMERICA.AL32UTF8
export NLS_DATE_FORMAT="YYYY-MM-DD HH24:MI:SS"
export TNS_ADMIN=$ORACLE_HOME/network/admin
export LANG=en_US

创建安装目录

1
2
3
4
5
6
7
$Oraclemkdir -p /u01/app/
$Oraclemkdir –p /u01/app/grid_base
$Oraclemkdir –p /u01/app/grid_home
$Oraclechown -R grid:oinstall /u01/app/grid_base
$Oraclechown -R grid:oinstall /u01/app/grid_home
$Oraclechown -R oracle:oinstall /u01/app
$Oraclechmod -R 775 /u01/app/

安装GRID

准备软件

软件上传至/u01/software
p13390677_112040_Linux-x86-64_1of7.zip
p13390677_112040_Linux-x86-64_2of7.zip
p13390677_112040_Linux-x86-64_3of7.zip

配置对等性

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
准备秘钥
$su - grid
$mkdir ~/.ssh
$ssh-keygen -t rsa
$ssh-keygen -t dsa

$su - oracle
$mkdir ~/.ssh
$ssh-keygen -t rsa
$ssh-keygen -t dsa

在节点1执行
$su - grid
$cat ~/.ssh/id_rsa.pub>>./.ssh/authorized_keys  --公钥存在authorized_keys文件中,写到本机
$cat ~/.ssh/id_dsa.pub>>./.ssh/authorized_keys
$ssh testOracle3Z-PRI cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys  --第二个节点的公钥写到本机
$ssh testOracle3Z-PRI cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
$scp ~/.ssh/authorized_keys testOracle3Z-PRI:~/.ssh/authorized_keys

$su - oracle
$cat ~/.ssh/id_rsa.pub>>./.ssh/authorized_keys  --公钥存在authorized_keys文件中,写到本机
$cat ~/.ssh/id_dsa.pub>>./.ssh/authorized_keys
$ssh testOracle3Z-PRI cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys  --第二个节点的公钥写到本机
$ssh testOracle3Z-PRI cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
$scp ~/.ssh/authorized_keys testOracle3Z-PRI:~/.ssh/authorized_keys

验证
$ssh rac1 date   (public网卡)
$ssh rac2 date
$ssh rac1-priv date  (private网卡)
$ssh rac2-priv date

安装预检查

1
2
3
$su - grid
$cd grid的软件目录
$./runcluvfy.sh stage -pre crsinst -n rac1,rac2 -verbose

准备响应文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
$vi crsinst 
ORACLE_HOSTNAME=rac1
INVENTORY_LOCATION=/u01/app/oraInventory
oracle.install.option=CRS_CONFIG
ORACLE_BASE=/u01/app/grid_base
ORACLE_HOME=/u01/app/grid_home
oracle.install.asm.OSDBA=asmdba
oracle.install.asm.OSOPER=asmoper
oracle.install.asm.OSASM=asmadmin
oracle.install.crs.config.gpnp.scanPort=1521
oracle.install.crs.config.clusterName=rac-cluster
oracle.install.crs.config.clusterNodes=rac1:rac1-vip,rac2:rac2-vip
oracle.install.crs.config.privateInterconnects=edge1:192.168.1.0:2,edge0:192.168.199.0:1   //1代表public，2代表private，3代表在群集中不使用该网卡
oracle.install.crs.config.storageOption=ASM_STORAGE
oracle.install.asm.SYSASMPassword=grid123456
oracle.install.asm.diskGroup.name=OCR
oracle.install.asm.diskGroup.redundancy=External
oracle.install.asm.diskGroup.AUSize=1
oracle.install.asm.diskGroup.disks=/dev/asm-diskb
oracle.install.asm.diskGroup.diskDiscoveryString=/dev/asm-disk*
oracle.install.asm.monitorPassword=grid

安装grid

1
2
3
4
$./runInstaller -silent -responseFile /u01/software/grid/response/crs_install_test.rsp -ignoreSysPrereqs -ignorePrereq
等待一段时间后，会提示在两个节点分别用root用户执行：
#/u01/grid/oraInventory/orainstRoot.sh
#/u01/crs/11.2.0/root.sh

创建磁盘组

1
2
3
4
5
#su – grid
$asmca
或者使用命令行
$ asmca -silent -createDiskGroup -sysAsmPassword grid12345 -diskString '/dev/asm-disk*' -diskGroupName DATA -diskList '/dev/asm-diskc' -redundancy External-compatible.asm 11.2 -compatible.rdbms 11.2 
$ asmca -silent -createDiskGroup -sysAsmPassword grid12345 -diskString '/dev/asm-disk*' -diskGroupName ARCH -diskList '/dev/asm-diskd' -redundancy External-compatible.asm 11.2 -compatible.rdbms 11.2

安装Database(图形安装可省略)

注册产品列表

/u01/app/oraInventory/ContentsXMLinventory.xml
/u01/app/oraInventory/ContentsXML

准备响应文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
$ vi /u01/soft/oracle/database/response/db_install_test.rsp
oracle.install.option=INSTALL_DB_SWONLY
ORACLE_HOSTNAME=rac1
UNIX_GROUP_NAME=oinstall
INVENTORY_LOCATION=/u01/app/oraInventory
SELECTED_LANGUAGES=en
ORACLE_HOME=/u01/app/oracle/product/11.2.0/db_1
ORACLE_BASE=/u01/app/oracle
oracle.install.db.InstallEdition=EE
oracle.install.db.isCustomInstall=false
oracle.install.db.optionalComponents=oracle.rdbms.partitioning:11.2.0.4.0,oracle.oraolap:11.2.0.4.0,oracle.rdbms.dm:11.2.0.4.0,oracle.rdbms.dv:11.2.0.4.0,oracle.rdbms.lbac:11.2.0.4.0,oracle.r
dbms.rat:11.2.0.4.0oracle.install.db.DBA_GROUP=dba
oracle.install.db.OPER_GROUP=oinstall
oracle.install.db.CLUSTER_NODES=rac1,rac2
DECLINE_SECURITY_UPDATES=true

安装

1
2
3
$./runInstaller -silent -ignoreSysPrereqs -ignorePrereq -responseFile /u01/software/database/response/db_install_test.rsp
#/u01/grid/oraInventory/orainstRoot.sh
#/u01/oracle/11.2.0/root.sh

创建数据库

创建数据库可以使用dbca交互界面，也可以使用静默创建方式

准备响应文件

1
2
3
4
5
6
7
8
9
10
$vi /u01/soft/oracle/database/response/dbca_orcl.rsp
GDBNAME = "wwlc"
SID = "wwlc"
NODELIST=rac1,rac2
SYSPASSWORD = "wwlc123456"
SYSTEMPASSWORD = "wwlc123456"
STORAGETYPE=ASM
DISKGROUPNAME=DATA
RECOVERYGROUPNAME=DATA
CHARACTERSET = "AL32UTF8"

创建数据库

1
dbca -silent -responseFile /u01/software/database/response/dbca_orcl.rsp

参考

阿里云上Oracle 11g RAC安装配置手册
手把手教你在ECS上部署Oracle RAC-博客-云栖社区-阿里云
如何在ECS上搭建Oracle？Oracle云上云下6种架构全解析-博客-云栖社区-阿里云