RHEL平台安装Oracle 12C+ RAC
单实例文件系统部署跳过GI和ASM配置部分即可
集群配置分类
- Oracle Standalone Clusters ：是本地化所有 Oracle Grid Infrastructure 服务和 Oracle ASM 的集群，并需要直接访问共享存储。
- Oracle Cluster Domain ：是 Oracle Clusterware 12c Release 2 中新引入的集群架构的部署选项。
- Oracle Member Clusters ：使用 Oracle 域服务集群的集中式服务，并可用于管理数据库或应用程序。
- Oracle Extended Clusters ：是由位于多个不同地理位置的多个节点组成的集群。
1. 检查清单
1.1 硬件条件
1）检查内存
# grep MemTotal /proc/meminfo
2）检查swap空间
grep SwapTotal /proc/meminfo
3）检查/tmp目录空间
df -h /tmp
4）检查共享内存（/dev/shm）
df -h /dev/shm
cluvfy Fails with Error:"PRVE-0427 : Failed To Retrieve The Size Of In-memory File System Mounted As /dev/shm " (文档 ID 2243758.1)
5）本地磁盘容量
创建软件安装目录
-
新建物理卷
[root@temp ~]# pvscan PV /dev/sda2 VG rootvg lvm2 [<63.50 GiB / 48.80 GiB free] Total: 1 [<63.50 GiB] / in use: 1 [<63.50 GiB] / in no VG: 0 [0 ] [root@temp ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 64G 0 disk ├─sda1 8:1 0 512M 0 part /boot └─sda2 8:2 0 63.5G 0 part ├─rootvg-rootlv 253:0 0 8G 0 lvm / ├─rootvg-swaplv 253:1 0 4G 0 lvm [SWAP] ├─rootvg-homelv 253:2 0 512M 0 lvm /home ├─rootvg-varlv 253:3 0 2G 0 lvm /var └─rootvg-usrlv 253:4 0 200M 0 lvm /usr/local sdb 8:16 0 64G 0 disk sr0 11:0 1 1024M 0 rom [root@temp ~]# pvcreate /dev/sdb Physical volume "/dev/sdb" successfully created. [root@temp ~]# pvscan PV /dev/sda2 VG rootvg lvm2 [<63.50 GiB / 48.80 GiB free] PV /dev/sdb lvm2 [64.00 GiB] Total: 2 [<127.50 GiB] / in use: 1 [<63.50 GiB] / in no VG: 1 [64.00 GiB] -
创建VG
[root@temp ~]# vgscan Reading volume groups from cache. Found volume group "rootvg" using metadata type lvm2 [root@temp ~]# vgcreate upsvg /dev/sdb Volume group "upsvg" successfully created [root@temp ~]# vgs VG #PV #LV #SN Attr VSize VFree rootvg 1 5 0 wz--n- <63.50g 48.80g upsvg 1 0 0 wz--n- <64.00g <64.00g [root@temp ~]# -
创建LV
[root@temp ~]# lvscan ACTIVE '/dev/rootvg/rootlv' [8.00 GiB] inherit ACTIVE '/dev/rootvg/swaplv' [4.00 GiB] inherit ACTIVE '/dev/rootvg/homelv' [512.00 MiB] inherit ACTIVE '/dev/rootvg/varlv' [2.00 GiB] inherit ACTIVE '/dev/rootvg/usrlv' [200.00 MiB] inherit [root@temp ~]# lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert homelv rootvg -wi-ao---- 512.00m rootlv rootvg -wi-ao---- 8.00g swaplv rootvg -wi-ao---- 4.00g usrlv rootvg -wi-ao---- 200.00m varlv rootvg -wi-ao---- 2.00g [root@temp ~]# [root@temp ~]# lvcreate -L 48G --name upslv upsvg Logical volume "upslv" created. [root@temp ~]# lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert homelv rootvg -wi-ao---- 512.00m rootlv rootvg -wi-ao---- 8.00g swaplv rootvg -wi-ao---- 4.00g usrlv rootvg -wi-ao---- 200.00m varlv rootvg -wi-ao---- 2.00g upslv upsvg -wi-a----- 48.00g [root@temp ~]# lvscan ACTIVE '/dev/upsvg/upslv' [48.00 GiB] inherit ACTIVE '/dev/rootvg/rootlv' [8.00 GiB] inherit ACTIVE '/dev/rootvg/swaplv' [4.00 GiB] inherit ACTIVE '/dev/rootvg/homelv' [512.00 MiB] inherit ACTIVE '/dev/rootvg/varlv' [2.00 GiB] inherit ACTIVE '/dev/rootvg/usrlv' [200.00 MiB] inherit [root@temp ~]# -
格式化
[root@temp ~]# mkfs.xfs /dev/upsvg/upslv meta-data=/dev/upsvg/upslv isize=512 agcount=4, agsize=3145728 blks = sectsz=512 attr=2, projid32bit=1 = crc=1 finobt=0, sparse=0 data = bsize=4096 blocks=12582912, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 ftype=1 log =internal log bsize=4096 blocks=6144, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 [root@temp ~]# -
挂载目录
mkdir -p /ups mount /dev/upsvg/upslv /ups cp /etc/fstab{,_$(date +%Y%m%d)} echo "/dev/mapper/upsvg-upslv /ups xfs defaults 0 0" >> /etc/fstab
6）存储容量需求
ASM存储平衡级别：
1.2 软件条件
2. 系统环境配置
2.1 rhel6系统配置
配置hosts文件
OPT_TIME=$(date +%Y%m%d)
cp /etc/host{,$OPT_TIME}
cat > /etc/hosts <<-EOF
127.0.0.1 loopback localhost.localdomain localhost
::1 loopback
# Public
192.168.10.166 node1
192.168.10.167 node1
# Private
172.168.0.156 node1-priv
172.168.0.157 node2-priv
# Virtual
192.168.10.168 node1-vip
192.168.10.169 node2-vip
# SCAN
192.168.10.170 orc12c-scan
192.168.10.171 orc12c-scan
192.168.10.172 orc12c-scan
EOF
安装系统软件包
cat > /tmp/os_pkgs.txt <<-EOF
bc
binutils
compat-libcap1
compat-libstdc++-33
compat-libstdc++-33.i686
e2fsprogs
e2fsprogs-libs
glibc.i686
glibc
glibc-devel.i686
glibc-devel
ksh
libaio
libaio.i686
libaio-devel
libaio-devel.i686
libX11.i686
libX11
libXau.i686
libXau
libXi.i686
libXi
libXtst.i686
libXtst
libgcc.i686
libgcc
libstdc++.i686
libstdc++
libstdc++-devel.i686
libstdc++-devel
libxcb.i686
libxcb
libXrender.i686
libXrender
libXrender-devel.i686
libXrender-devel
make
net-tools
nfs-utils
smartmontools
sysstat
libXext
gcc
gcc-c++
# --
cloog-ppl
cpp
glibc-headers
kernel-headers
mpfr
ppl
EOF
# while read line; do
# rpm -q --queryformat "%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})
" $line
# done < /tmp/os_pkgs.txt
WORK_DIR='/tmp'
OS_YUM_PACKAGES=$(cat /tmp/os_pkgs.txt)
rpm -qa --queryformat "%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})
" ${OS_YUM_PACKAGES}
if [[ "$?" != "0" ]]; then
# 缺依赖包，需安装
[[ -z "${ISO_FILE}" ]] && ISO_FILE=$(ls "${WORK_DIR}"/*.iso|awk -F"/" 'NR==1{print $NF}')
if [[ -f "${ISO_FILE}" ]]; then
# 缺ISO文件则挂载cdrom，否则挂载ISO
mount -o loop ${ISO_FILE} /mnt
else
mount -o loop /dev/cdrom /mnt
fi
if [[ -f "/mnt/RPM-GPG-KEY-redhat-release" ]]; then
# 挂载成功则安装依赖包
${CP} -rpf /etc/yum.repos.d ${MODULE_BACKUP_DIR}
echo -e '[Server]
name=LocalYUM' > /etc/yum.repos.d/oracle.repo
echo 'baseurl=file:///mnt' >> /etc/yum.repos.d/oracle.repo
echo -e 'enabled=1
gpgcheck=0' >> /etc/yum.repos.d/oracle.repo
echo 'gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release' >> /etc/yum.repos.d/oracle.repo
yum -y install ${OS_YUM_PACKAGES}
umount /mnt
${RM} -f "/etc/yum.repos.d/oracle.repo"
else
# 挂载失败则使用系统自带的yum配置进行安装
yum -y install ${OS_YUM_PACKAGES}
fi
fi
-- 检查确认
for i in $(cat /tmp/os_pkgs.txt); do
rpm -q --queryformat "%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})
" ${i} |grep 'is not installed'
done
关闭selinux
if [[ "$(getenforce)" = "Enforcing" ]]; then
cp /etc/selinux/config{,$OPT_TIME}
setenforce 0
sed -i "/^SELINUX=enforcing/c#SELINUX=enforcing
SELINUX=disable" /etc/selinux/config
fi
关闭防火墙
chkconfig iptables off
service iptables stop
配置系统参数
# the value of size in Kb
MEMTOTAL=$(grep MemTotal /proc/meminfo|awk '{print $2}')
SHMMAX=$(( MEMTOTAL * 1024 * 8 / 10 ))
SHMMNI=4096
PAGESIZE=$(getconf PAGE_SIZE)
SHMALL=$(( SHMMAX / 4096 ))
cat > /etc/sysctl.d/97-oracle-database-sysctl.conf<<-EOF
# -- The number of asynchronous IO requests at the same time(as per Note 579108.1),for example 1048576 = 1024 * 1024
fs.aio-max-nr = 3145728
# -- 512 * processes (for example 6815744 for 13312 processes)
fs.file-max = 6815744
# is maximum of sga in bytes
kernel.shmmax = ${SHMMAX}
kernel.shmall = ${SHMALL}
kernel.shmmni = ${SHMMNI}
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
net.ipv4.tcp_rmem = 4096 87380 4194304
net.ipv4.tcp_wmem = 4096 16384 4194304
kernel.panic_on_oops = 1
vm.min_free_kbytes = 524288
EOF
/sbin/sysctl -p /etc/sysctl.d/98-oracle.conf
配置用户资源限制
修改完成后保存, 退出当前用户并重新登录(不用重启服务器), 当前修改就会生效
MEMTOTAL=$(grep MemTotal /proc/meminfo|awk '{print $2}')
MEMLOCK=$(( MEMTOTAL * 9/10 ))
cat > /etc/security/limits.d/99-grid-oracle-limits.conf << EOF
oracle soft nproc 16384 #Ora bug 15971421
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
oracle soft stack 10240
oracle hard stack 32768
# setting for memlock hard limit is maximum of 128GB on x86_64 or 3GB on x86 OR 90 % of RAM
oracle soft memlock ${MEMLOCK}
oracle hard memlock ${MEMLOCK}
grid soft nproc 16384 # Ora bug 15971421
grid hard nproc 16384
grid soft nofile 1024
grid hard nofile 65536
grid soft stack 10240
grid hard stack 32768
grid soft memlock ${MEMLOCK}
grid hard memlock ${MEMLOCK}
EOF
参数说明
# 1. nproc 默认值的计算方法
# 计算公式为:
default_nproc = max_threads / 2;
# 其中, max_threads = mempages / (8 * THREAD_SIZE / PAGE_SIZE);
# mempages是机器的物理页面个数, THREAD_SIZE=8K, 所以, 计算公式为:
default_nproc = max_threads / 2
= (mempages * PAGE_SIZE) / ( 2 * 8 * THREAD_SIZE )
= total_memory / 128K;
# 计算本机默认nproc配置:
cat /proc/meminfo | grep MemTotal
MemTotal: 115571480 kB
echo "115571480 / 128" | bc
902902
ulimit -u
902682
# 再rhel6.8中default_nproc == (total_memory / 128K / 2)
[root@appServer config]# echo $(($(awk '/MemTotal/{print $2}' /proc/meminfo ) /256))
15298
[root@appServer config]# ulimit -u
15187
[root@appServer config]#
# ulimit -a 命令的输出信息:
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited # 一个进程的数据段的最大值
scheduling priority (-e) 0
file size (blocks, -f) unlimited # Shell创建文件的最大体积, 1block = 512bytes
pending signals (-i) 1031426 # 最多允许多少个待处理的信号
max locked memory (kbytes, -l) 64 # 每个进程可以锁住的物理内存的最大值
max memory size (kbytes, -m) unlimited # 每个进程可以使用的常驻内存的最大值
open files (-n) 65535 # 每个进程可以同时打开的最大文件数, 不能是unlimited
pipe size (512 bytes, -p) 8 # 管道的最大值, 1block = 512bytes
POSIX message queues (bytes, -q) 819200 # POSIX的消息队列的最大值
real-time priority (-r) 0
stack size (kbytes, -s) 10240 # 单个进程能够使用的最大栈大小
cpu time (seconds, -t) unlimited # 单个进程的最大CPU时间, 也就是可使用CPU的秒数, 到硬极限时, 这个进程就会立即自杀; 到软极限时, 每秒发送一次限制超时信号SIGXCPU
max user processes (-u) 131072 # 单个用户可同时运行的最大进程数, 不能是unlimited
virtual memory (kbytes, -v) unlimited # 每个进程可使用的最大虚拟内存
file locks (-x) unlimited # 每个进程能锁住的最大文件个数
-H 设置某个给定资源的硬极限. 如果用户拥有root权限, 可以增大硬极限. 任何用户均可减少硬极限
-S 设置某个给定资源的软极限, 软极限可增大到硬极限的值
创建用户
# 创建用户组
groupadd --gid 54321 oinstall
groupadd --gid 54322 dba
groupadd --gid 54323 asmdba
groupadd --gid 54324 asmoper
groupadd --gid 54325 asmadmin
groupadd --gid 54326 oper
groupadd --gid 54327 backupdba
groupadd --gid 54328 dgdba
groupadd --gid 54329 kmdba
groupadd --gid 54330 racdba
# 创建用户
useradd --uid 54321 --gid oinstall --groups dba,oper,asmdba,asmoper,backupdba,dgdba,kmdba,racdba --comment "Oracle Software Owner" oracle
useradd --uid 54322 --gid oinstall --groups dba,asmadmin,asmdba,asmoper --comment "Grid Infrastructure Owner" grid
# 设置用户密码
OS_PASS=123456
echo "${OS_PASS}"|passwd --stdin oracle
echo "${OS_PASS}"|passwd --stdin grid
创建软件目录
mkdir -p /oracle/app/grid
mkdir -p /oracle/app/12.2/grid
mkdir -p /oracle/app/oraInventory
chown -R grid:oinstall /oracle/app
mkdir -p /oracle/app/database
chown oracle:oinstall /oracle/app/database
chmod -R 775 /oracle/app
编辑用户环境变量
注意：根据实际修改个节点ORACLE_SID值
su - grid
vi ~/.bash_profile
export ORACLE_SID=+ASM1
export ORACLE_BASE=/oracle/app/grid
export ORACLE_HOME=/oracle/app/12.2/grid
export PATH=${ORACLE_HOME}/bin:${ORACLE_HOME}/OPatch:${PATH}
umask 022
su - oracle
vi ~/.bash_profile
export ORACLE_SID=orcl1
export ORACLE_BASE=/oracle/app/database
export ORACLE_HOME=${ORACLE_BASE}/product/db_1
export GRID_HOME=/oracle/app/12.2/grid
export LD_LIBRARY_PATH=${ORACLE_HOME}/lib
export PATH=${PATH}:${ORACLE_HOME}/bin:${ORACLE_HOME}/OPatch:${ORACLE_HOME}/suptools/oratop:${GRID_HOME}/bin
export TNS_ADMIN=${GRID_HOME}/network/admin
umask 022
对于 12cR2 以后的 Oracle 版本， oratop位于文件夹 ${ORACLE_HOME}/suptools/oratop 下。此外， 它还与 Oracle 跟踪文件分析器（TFA） 捆绑在一起
配置profile
cat > /etc/profile.d/oracle-grid.sh << EOF
#Setting the appropriate ulimits for oracle and grid user
if [ $USER = "oracle" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -u 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi
if [ $USER = "grid" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -u 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi
EOF
关闭 transparent_hugepage
# rhel6
TH_CONF='/etc/grub.conf'
# rhel7 /etc/default/grub
if [[ -f "${TH_CONF}" ]] ; then
cp -p ${TH_CONF}{,$OPT_TIME}
sed -ri 's/^(GRUB_CMDLINE_LINUX=.*quiet)"$/1 transparent_hugepage=never"/' ${TH_CONF}
fi
配置认证模块
cp /etc/pam.d/login{,$OPT_TIME}
cat >> /etc/pam.d/login <<-EOF
session required pam_limits.so
EOF
配置NTP
# 查看当前时区
cat /etc/sysconfig/clock
# 添加ntp服务器，根据实际环境修改服务地址
vi /etc/ntp.conf
server 127.127.0.1
# 配置ntpd的参数，我们主要强调的是要配置成"微调的模式" 也就是在options中要加入-x的选项
vi /etc/sysconfig/ntpd
OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid"
# 开机启动NTPD服务
chkconfig ntpd on
# 重启服务，使最新配置生效
service ntpd restart
# 检查是否生效
ntpq -p
ps -ef |grep ntp|grep -v grep
# 安装完成后需要使用如下命令确认
$ crsctl check ctss
# 若当前环境没有NTP服务器时，将/etc/resolv.conf文件移除或重命名。以便使用Oracle ctss服务同步集群事件。
配置/dev/shm(AMM内存管理)
在Linux环境中，该配置会影响数据库automatic memory management机制。因此，在AMM的环境中需要确认/dev/shm可用且其值大于MEMORY_TARGET。所有的SGA 内存都是在/dev/shm 下分配。
cat /etc/fstab
tmpfs /dev/shm tmpfs defaults,size=337920M 0 0
mount -o remount /dev/shm
配置SSH互信
export SSH='ssh -o PasswordAuthentication=no -o StrictHostKeyChecking=no'
ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
cat ~/.ssh/*.pub >> ~/.ssh/authorized_keys
ssh -o stricthostkeychecking=no node2 cat ~/.ssh/*.pub >> ~/.ssh/authorized_keys
scp ~/.ssh/authorized_keys node2:~/.ssh/authorized_keys
scp -r ~/.ssh /home/grid/
scp -r ~/.ssh /home/oracle/
chown grid:oinstall /home/grid/.ssh -R
chown oracle:oinstall /home/oracle/.ssh -R
# 检查确认
export SSH='ssh -o ConnectTimeout=3 -o ConnectionAttempts=5 -o PasswordAuthentication=no -o StrictHostKeyChecking=no'
for hosts in $(grep -Ev "^#|-vip|scan" /etc/hosts|awk '{print $NF}'); do
${SSH} ${hosts} date
done
2.2 rhel7系统配置
配置虚拟文件系统
vi /etc/fstab
tmpfs /dev/shm tmpfs defaults,rw,exec,size=2G 0 0
-- 重新装载
# mount -o remount /dev/shm
disable Transparent HugePages
# 检查
cat /sys/kernel/mm/transparent_hugepage/enabled
1. For Oracle Linux 7 and Red Hat Enterprise Linux 7, add or modify the transparent_hugepage=never parameter
in the /etc/default/grub file:transparent_hugepage=never
For example:
RUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="crashkernel=auto quiet numa=off transparent_hugepage=never"
GRUB_DISABLE_RECOVERY="true"
# fdisl -l /dev/sda 检查确认磁盘分区类型
# 备份配置文件
## For an MBR (BIOS-based) system:
cp /etc/default/grub /etc/default/grub-backup
cp /boot/grub2/grub.cfg /boot/grub2/grub.cfg-backup
## For a GPT (UEFI-based) system
cp /etc/default/grub /etc/default/grub-backup
cp /boot/efi/EFI/redhat/grub.cfg /boot/efi/EFI/redhat/grub.cfg-backup
2. Run the grub2–mkconfig command to regenerate the grub.cfg file.
## Please ensure to take a backup of the existing /boot/grub2/grub.cfg before rebuilding.
On BIOS-based machines: ~]# grub2-mkconfig -o /boot/grub2/grub.cfg
On UEFI-based machines: ~]# grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg
3. Restart the system to make the changes permanent.
systemctl reboot
# 4. 检查确认
cat /proc/cmdline
cat /sys/kernel/mm/transparent_hugepage/enabled
若仍无法关闭时
If Transparent Huge Pages (THP) is still not disabled, continue and use one of the options below.
Option 1: (Recommended) create a customized tuned profile with disabled THP
With this resolution we will create a customized version of the currently running profile. The customized version will disable THP.
Find out which profile is active, create a copy. In the following example we currently use the throughput-performance profile:
# tuned-adm active
Current active profile: throughput-performance
To create customized profile, create a new directory in /etc/tuned directory with desired profile name.
# mkdir /etc/tuned/myprofile-nothp
Then create a new tuned.conf file for myprofile-nothp, and insert the new tuning info:
# cat /etc/tuned/myprofile-nothp/tuned.conf
[main]
include=throughput-performance
[vm]
transparent_hugepages=never
Make the script executable:
# chmod +x /etc/tuned/myprofile-nothp/tuned.conf
Enable myprofile like so:
# tuned-adm profile myprofile-nothp
This change will immediately take effect and persist reboots.
To verify if THP are disabled or not, run below command:
# cat /sys/kernel/mm/transparent_hugepage/enabled
Option 2: (Alternative) Disable tuned services
This resolution will disable the tuned services.
# systemctl stop tuned
# systemctl disable tuned
OR
# tuned-adm off
Now add "transparent_hugepage=never" kernel parameter in grub2 configuration file as explained in steps 1-3 above.
Reboot the server for changes to take effect.
安装系统 rpm包
-- Packages for Red Hat Enterprise Linux 7:
gcc
gcc-c++
bc
binutils
compat-libcap1
compat-libstdc++-33
dtrace-modules
dtrace-modules-headers
dtrace-modules-provider-headers
dtrace-utils
elfutils-libelf
elfutils-libelf-devel
fontconfig-devel
glibc
glibc-devel
ksh
libaio
libaio-devel
libdtrace-ctf-devel
libX11
libXau
libXi
libXtst
libXrender
libXrender-devel
libgcc
librdmacm-devel
libstdc++
libstdc++-devel
libxcb
make
net-tools (for Oracle RAC and Oracle Clusterware)
nfs-utils (for Oracle ACFS)
python (for Oracle ACFS Remote)
python-configshell (for Oracle ACFS Remote)
python-rtslib (for Oracle ACFS Remote)
python-six (for Oracle ACFS Remote)
targetcli (for Oracle ACFS Remote)
smartmontools
sysstat
unixODBC
unixODBC-devel
rpm -q --queryformat "%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})
" gcc gcc-c++ bc binutils compat-libcap1 compat-libstdc++-33 dtrace-modules dtrace-modules-headers dtrace-modules-provider-headers dtrace-utils elfutils-libelf elfutils-libelf-devel fontconfig-devel glibc glibc-devel ksh libaio libaio-devel libdtrace-ctf-devel libX11 libXau libXi libXtst libXrender libXrender-devel libgcc librdmacm-devel libstdc++ libstdc++-devel libxcb make net-tools smartmontools sysstat unixODBC unixODBC-devel
yum -y install compat-libstdc++-33.i686 glibc.i686 glibc-devel.i686 libaio.i686 libaio-devel.i686 libgcc.i686 libstdc++.i686 libstdc++-devel.i686 libXi.i686 libXtst.i686 unixODBC.i686 unixODBC-devel.i686
yum -y install gcc gcc-c++ bc binutils compat-libcap1 compat-libstdc++-33 dtrace-modules dtrace-modules-headers dtrace-modules-provider-headers dtrace-utils elfutils-libelf elfutils-libelf-devel fontconfig-devel glibc glibc-devel ksh libaio libaio-devel libdtrace-ctf-devel libX11 libXau libXi libXtst libXrender libXrender-devel libgcc librdmacm-devel libstdc++ libstdc++-devel libxcb make net-tools smartmontools sysstat unixODBC unixODBC-devel
配置/etc/hosts文件
cp /etc/hosts /etc/hosts_$(date +%Y%d%m)
cat > /etc/hosts << EOF
127.0.0.1 loopback localhost.localdomain localhost localhost4
::1 loopback localhost.localdomain localhost localhost6
# Public
192.168.10.173 o19c1 o19c1.example.com
192.168.10.174 o19c2 o19c2.example.com
# Private
172.168.0.158 o19c1-priv o19c1.example.com-priv
172.168.0.159 o19c2-priv o19c2.example.com-priv
# Virtual
192.168.10.175 o19c1-vip o19c1.example.com-vip
192.168.10.176 o19c2-vip o19c2.example.com-vip
# SCAN
#192.168.10.177 o19c-scan o19c-scan.example.com
#192.168.10.178 o19c-scan o19c-scan.example.com
#192.168.10.179 o19c-scan o19c-scan.example.com
EOF
ssh配置 (LoginGraceTime 0)
cp /etc/ssh/sshd_config /etc/ssh/sshd_config_$(date +%Y%m%d)
or
cp /etc/ssh/sshd_config{,_$(date +%Y%m%d)}
grep '^LoginGraceTime' /etc/ssh/sshd_config
echo "LoginGraceTime 0" >> /etc/ssh/sshd_config
关闭selinux
if [[ "$(getenforce)" = "Enforcing" ]]; then
cp /etc/selinux/config /etc/selinux/config_$(date +%Y%m%d)
setenforce 0
# sed -i "/^SELINUX=enforcing/c#SELINUX=enforcing
SELINUX=disable" /etc/selinux/config
sed -i "/^SELINUX=enforcing/aSELINUX=disable" /etc/selinux/config
fi
关闭防火墙
if [[ "$(ps -ef |grep -v grep|grep -ci firewalld)" = "1" ]]; then
systemctl stop firewalld
systemctl disable firewalld
fi
配置PAM
vi /etc/pam.d/login
session required pam_limits.so
cp /etc/pam.d/login{,_$(date +%Y%m%d)}
echo "session required pam_limits.so" >> /etc/pam.d/login
禁用avahi-daemon服务
--查看服务状态
systemctl status avahi-daemon
--停止服务
systemctl stop avahi-daemon
--禁止服务自启动
systemctl disable avahi-daemon
if [[ -f "/etc/systemd/system/dbus-org.freedesktop.Avahi.service" ]]; then
systemctl stop avahi-dnsconfd
systemctl stop avahi-daemon
systemctl disable avahi-dnsconfd
systemctl disable avahi-daemon
fi
屏蔽RemoveIPC参数 [ rhel 7.2 BUG Doc ID 2081410.1 ]
if [[ -f "/etc/systemd/logind.conf" ]]; then
cp /etc/systemd/logind.conf /etc/systemd/logind.conf_$(date +%Y%m%d)
sed -i "/#RemoveIPC=/aRemoveIPC=no" /etc/systemd/logind.conf
fi
grep '^RemoveIPC' /etc/systemd/logind.conf
[[ "$?" -eq "1" ]] && cp /etc/systemd/logind.conf{,_$(date +%Y%m%d)} && sed -i "/#RemoveIPC=/aRemoveIPC=no" /etc/systemd/logind.conf
# 重启服务器或重启systemd-logind
systemctl daemon-reload
systemctl restart systemd-logind
配置系统资源限制
# the value of size in Kb
MEMTOTAL=$(grep MemTotal /proc/meminfo|awk '{print $2}')
MEMLOCK=$(( MEMTOTAL * 9/10 ))
cat > /etc/security/limits.d/99-grid-oracle-limits.conf << EOF
oracle soft nproc 16384 #Ora bug 15971421
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
oracle soft stack 10240
oracle hard stack 32768
# setting for memlock hard limit is maximum of 128GB on x86_64 or 3GB on x86 OR 90 % of RAM
oracle soft memlock ${MEMLOCK}
oracle hard memlock ${MEMLOCK}
grid soft nproc 16384 #Ora bug 15971421
grid hard nproc 16384
grid soft nofile 1024
grid hard nofile 65536
grid soft stack 10240
grid hard stack 32768
grid soft memlock ${MEMLOCK}
grid hard memlock ${MEMLOCK}
EOF
配置系统内核参数
# the value of size in Kb
MEMTOTAL=$(grep MemTotal /proc/meminfo|awk '{print $2}')
SHMMAX=$(( MEMTOTAL * 1024 * 8 / 10 ))
SHMMNI=4096
PAGESIZE=$(getconf PAGE_SIZE)
SHMALL=$(( SHMMAX / 4096 ))
cat > /etc/sysctl.d/97-oracle-database-sysctl.conf << EOF
# -- The number of asynchronous IO requests at the same time(as per Note 579108.1),for example 1048576 = 1024 * 1024
fs.aio-max-nr = 3145728
# -- 512 * processes (for example 6815744 for 13312 processes)
fs.file-max = 6815744
# is maximum of sga in bytes
kernel.shmmax = ${SHMMAX}
kernel.shmall = ${SHMALL}
kernel.shmmni = ${SHMMNI}
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
net.ipv4.tcp_rmem = 4096 87380 4194304
net.ipv4.tcp_wmem = 4096 16384 4194304
kernel.panic_on_oops = 1
vm.min_free_kbytes = 524288
EOF
# /sbin/sysctl --system
-- 检查确认
/sbin/sysctl -a |grep net.core.[wr]mem_max
Table A-1 Minimum Operating System Resource Parameter Settings
| Parameter | Value | File |
|---|---|---|
| semmsl semmns semopm semmni |
250 32000 100 128 |
/proc/sys/kernel/sem |
| shmall | Greater than or equal to the value of shmmax, in pages. | /proc/sys/kernel/shmall |
| shmmax | Half the size of physical memory in bytesSee My Oracle Support Note 567506.1 for additional information about configuring shmmax. | /proc/sys/kernel/shmmax |
| shmmni | 4096 | /proc/sys/kernel/shmmni |
| panic_on_oops | 1 | /proc/sys/kernel/panic_on_oops |
| file-max | 6815744 | /proc/sys/fs/file-max |
| aio-max-nr | 1048576Note: This value limits concurrent outstanding requests and should be set to avoid I/O subsystem failures. | /proc/sys/fs/aio-max-nr |
| ip_local_port_range | Minimum: 9000Maximum: 65500 | /proc/sys/net/ipv4/ip_local_port_range |
| rmem_default | 262144 | /proc/sys/net/core/rmem_default |
| rmem_max | 4194304 | /proc/sys/net/core/rmem_max |
| wmem_default | 262144 | /proc/sys/net/core/wmem_default |
| wmem_max | 1048576 | /proc/sys/net/core/wmem_max |
UDP和TCP 内核参数配置(临时)
# 检查
cat /proc/sys/net/ipv4/ip_local_port_range
# 配置
echo 9000 65500 > /proc/sys/net/ipv4/ip_local_port_range
#
/etc/rc.d/init.d/network restart
配置网络参数nozeroconf
cp /etc/sysconfig/network /etc/sysconfig/network_$(date +%Y%m%d)
cat >> /etc/sysconfig/network << EOF
NOZEROCONF=yes
EOF
cat >> /etc/sysconfig/network-scripts/ifcfg-lo << EOF
MTU=16436
EOF
系统时间校对
RHEL建议使用NTP [Tips on Troubleshooting NTP / chrony Issues](Doc ID 2068875.1)
# 检验时间和时区确认正确
date
# 查看当前时区
timedatectl status
# 修改时区（若需要）
timedatectl set-timezone Asia/ShangHai
# 关闭chrony服务，移除chrony配置文件（虚机不配置时间同步服务，使用ctss）
systemctl list-unit-files|grep chronyd
systemctl status chronyd
systemctl disable chronyd
systemctl stop chronyd
# 移除方式备份chrony配置文件
mv /etc/chrony.conf /etc/chrony.conf_bak
# chronyd服务配置时间同步
# 配置时间服务器（若需要）
vi /etc/chrony.conf
server 192.168.10.166 iburst
# 检查确认
chronyc sources -v
chronyc sourcestats -v
# NTP服务方式配置同步时间
# 1. 关闭并禁用chrony服务
systemctl list-unit-files|grep chronyd
systemctl status chronyd
systemctl disable chronyd
systemctl stop chronyd
# 2. 删除其配置文件
mv /etc/chrony.conf /etc/chrony.conf_bak
# 3. 安装NTP服务
yum install -y ntp ntpdate
systemctl enable ntpd
systemctl start ntpd
# 手动同步时间
ntpdate -u <NTP Server>
# 4. 配置NTP，开启微调模式
# 编辑/etc/sysconfig/ntpd，在-g后面加上-x 和 -p参数
# Command line options for ntpd
OPTIONS="-g -x -p /var/run/ntpd.pid"
# 5. 检查确认
ntpq -p
ntpstat # 查看这台服务器是否连接到NTP服务器
创建用户组和用户
# 1. 创建用户组
/usr/sbin/groupadd -g 54321 oinstall
/usr/sbin/groupadd -g 54322 dba
/usr/sbin/groupadd -g 54323 oper
/usr/sbin/groupadd -g 54324 backupdba
/usr/sbin/groupadd -g 54325 dgdba
/usr/sbin/groupadd -g 54326 kmdba
/usr/sbin/groupadd -g 54327 asmdba
/usr/sbin/groupadd -g 54328 asmoper
/usr/sbin/groupadd -g 54329 asmadmin
/usr/sbin/groupadd -g 54330 racdba
# 2. 创建用户
/usr/sbin/useradd -u 54321 -g oinstall -G dba,oper,asmdba,backupdba,dgdba,kmdba,racdba --comment "Oracle Software Owner" oracle
/usr/sbin/useradd -u 54322 -g oinstall -G asmadmin,asmdba,asmoper,dba --comment "Oracle Software Owner" grid
$ id oracle
uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba), 54323(oper),54324(backupdba),54325(dgdba),54326(kmdba),54327(asmdba),54330(racdba)
$ id grid
uid=54331(grid) gid=54321(oinstall) groups=54321(oinstall),54322(dba),54327(asmdba),54328(asmoper),54329(asmadmin),54330(racdba)
# 3. 配置密码
echo "oracle"|passwd --stdin oracle
echo "grid"|passwd --stdin grid
创建软件安装目录
# 1. 软件安装目录
mkdir -p /ups/oracle/grid
mkdir -p /ups/oracle/19c/grid
mkdir -p /ups/oracle/oraInventory
chown -R grid:oinstall /ups/oracle
mkdir -p /ups/oracle/database/db_1
chown -R oracle:oinstall /ups/oracle/database
chmod -R 775 /ups/oracle
# 安装文件目录（临时存储）
mkdir -p /ups/soft
chown -R grid:oinstall /ups/soft
chmod 775 /ups/soft
#
cat >> /etc/oraInst.loc <<EOF
inventory_loc=/ups/oracle/oraInventory
inst_group=oinstall
EOF
chown grid:oinstall /etc/oraInst.loc
chmod 644 /etc/oraInst.loc
配置用户环境变量
su - grid
vi ~/.bash_profile
export ORACLE_SID=+ASM1
export ORACLE_BASE=/ups/oracle/grid
export ORACLE_HOME=/ups/oracle/19c/grid
export PATH=${ORACLE_HOME}/bin:${ORACLE_HOME}/OPatch:${PATH}
export INVENTORY_LOCATION=/ups/oracle/oraInventory
umask 022
if [ -t 0 ]; then
stty intr ^C
fi
su - oracle
vi ~/.bash_profile
export ORACLE_SID=o19cdb1
export ORACLE_BASE=/ups/oracle/database
export ORACLE_HOME=${ORACLE_BASE}/db_1
export GRID_HOME=/ups/oracle/19c/grid
export LD_LIBRARY_PATH=${ORACLE_HOME}/lib
export PATH=${ORACLE_HOME}/bin:${ORACLE_HOME}/OPatch:${ORACLE_HOME}/suptools/oratop:${ORACLE_HOME}/perl/bin:${GRID_HOME}/bin:${HOME}/scripts/bin:${PATH}
export TNS_ADMIN=${GRID_HOME}/network/admin
export SQLPATH=${ORACLE_HOME}/rdbms/admin:${HOME}/scripts/sql
umask 022
if [ -t 0 ]; then
stty intr ^C
fi
配置profile
cat > /etc/profile.d/oracle-grid.sh << EOF
#Setting the appropriate ulimits for oracle and grid user
if [ $USER = "oracle" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -u 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi
if [ $USER = "grid" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -u 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi
EOF
配置ssh互相
export SSH='ssh -o PasswordAuthentication=no -o StrictHostKeyChecking=no'
ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
cat ~/.ssh/*.pub >> ~/.ssh/authorized_keys
ssh -o stricthostkeychecking=no 192.168.10.174 cat ~/.ssh/*.pub >> ~/.ssh/authorized_keys
scp ~/.ssh/authorized_keys 192.168.10.174:~/.ssh/authorized_keys
scp -r ~/.ssh /home/grid/
scp -r ~/.ssh /home/oracle/
chown grid:oinstall /home/grid/.ssh -R
chown oracle:oinstall /home/oracle/.ssh -R
# 检查确认
export SSH='ssh -o ConnectTimeout=3 -o ConnectionAttempts=5 -o PasswordAuthentication=no -o StrictHostKeyChecking=no'
for ip in $(grep -Ev "^#|localhost|vip|scan" /etc/hosts); do
${SSH} ${ip} date
done
# 另一个方式使用Oracle自带脚本创建
配置dns
# yum 安装
yum -y install unbound
yum install -y bind-utils
# 配置配置文件/etc/unbound/unbound.conf
vi /etc/unbound/unbound.conf
……
38 # interface: 0.0.0.0
39 interface: 0.0.0.0
……
//找到38行，复制去掉注释行，打开监听全网功能。
177 # access-control: 0.0.0.0/0 refuse
178 access-control: 192.168.10.0/24 allow
179 # access-control: 127.0.0.0/8 allow
// 找到配置文件/etc/unbound/unbound.conf的第177行，缺省为注释行，且内容为拒绝访问。复制本行内容到下面一行，去掉注释“#“，改refuse为allow。然后保存退出，重启服务即可。
155 # do-ip6: yes
156 do-ip6: no
//找到155行内容，在其下复制一行并去除注释，改yes为no，重启服务即可去除对Ipv6的监听
# 创建解析文件
cat > /etc/unbound/local.d/example.conf << EOF
local-zone: "example.com." static
local-data: "example.com. 86400 IN SOA ns.example.com. root 1 1D 1H 1W 1H"
local-data: "ns.example.com. IN A 192.168.10.173"
local-data: "o19c1.example.com. IN A 192.168.10.173"
local-data: "o19c-scan.example.com. IN A 192.168.10.177"
local-data: "o19c-scan.example.com. IN A 192.168.10.178"
local-data: "o19c-scan.example.com. IN A 192.168.10.179"
local-data-ptr: "192.168.10.177 o19c-scan.example.com."
local-data-ptr: "192.168.10.178 o19c-scan.example.com."
local-data-ptr: "192.168.10.179 o19c-scan.example.com."
EOF
# 启动服务及检查
systemctl start unbound
systemctl restart unbound
systemctl status unbound
netstat -tunlp |grep unbound
ss -tunlp|grep unbound
cat /etc/resolv.conf
# Generated by NetworkManager
search example.com
nameserver 192.168.10.173
-- 验证
nslookup o19c-scan
[root@o19c2 network-scripts]# nslookup o19c-scan
Server: 192.168.10.173
Address: 192.168.10.173#53
Name: o19c-scan.example.com
Address: 192.168.10.178
Name: o19c-scan.example.com
Address: 192.168.10.179
Name: o19c-scan.example.com
Address: 192.168.10.177
# 若不启用DNS，则执行下面命令
mv /etc/resolv.conf /etc/resolv.conf_orig
关闭automatic NUMA balancing
# 检查 Automatic NUMA balancing (0:未启用， 1:已启用)
sysctl -e kernel.numa_balanceing
# 关闭 automatic NUMA memory balancing 特性
cat >> /etc/sysctl.d/97-oracle-database-sysctl.conf <<-EOF
kernel.numa_balancing = 0
EOF
2.3 配置共享存储
虚拟机创建共享磁盘
# vmware 配置
set path=%path%;C:UpgsVMwareVMware Workstation
vmware-vdiskmanager.exe -c -s 2g -a lsilogic -t 4 "o19c-crs1.vmdk"
vmware-vdiskmanager.exe -c -s 2g -a lsilogic -t 4 "o19c-crs2.vmdk"
vmware-vdiskmanager.exe -c -s 2g -a lsilogic -t 4 "o19c-crs3.vmdk"
vmware-vdiskmanager.exe -c -s 32g -a lsilogic -t 4 "o19c-mgmt.vmdk"
vmware-vdiskmanager.exe -c -s 24g -a lsilogic -t 4 "o19c-data.vmdk"
vmware-vdiskmanager.exe -c -s 8g -a lsilogic -t 4 "o19c-data1.vmdk"
-- 编辑o19c1.vmx和o19c2.vmx文件，可以使用uuid
#
# ----------------------------------------------------------------
# SHARED DISK SECTION - (BEGIN)
# ----------------------------------------------------------------
# - The goal in meeting the hardware requirements is to have a
# shared storage for the two nodes. The way to achieve this in
# VMware is the creation of a NEW SCSI BUS. It has to be of
# type "virtual" and we must have the disk.locking = "false"
# option.
# - Just dataCacheMaxSize = "0" should be sufficient with the
# diskLib.* parameters, although I include all parameters for
# documentation purposes.
# - maxUnsyncedWrites should matter for sparse disks only, and
# I certainly do not recommend using sparse disks for
# clustering.
# - dataCacheMaxSize=0 should disable cache size completely, so
# other three dataCache options should do nothing (no harm,
# but nothing good either).
# ----------------------------------------------------------------
#
diskLib.dataCacheMaxSize = "0"
diskLib.dataCacheMaxReadAheadSize = "0"
diskLib.dataCacheMinReadAheadSize = "0"
diskLib.dataCachePageSize = "4096"
diskLib.maxUnsyncedWrites = "0"
disk.locking = "false"
# ----------------------------------------------------------------
# Create one HBA
# ----------------------------------------------------------------
scsi1.present = "TRUE"
scsi1.virtualDev = "lsilogic"
scsi1.sharedBus = "virtual"
scsi1:0.present = "TRUE"
scsi1:0.present = "TRUE"
scsi1:0.fileName = "..asmdisko19cdb-crs1.vmdk"
scsi1:0.mode = "independent-persistent"
scsi1:0.redo = ""
scsi1:1.present = "TRUE"
scsi1:1.fileName = "..asmdisko19cdb-crs2.vmdk"
scsi1:1.mode = "independent-persistent"
scsi1:1.redo = ""
scsi1:2.present = "TRUE"
scsi1:2.fileName = "..asmdisko19cdb-crs3.vmdk"
scsi1:2.mode = "independent-persistent"
scsi1:2.redo = ""
scsi1:3.present = "TRUE"
scsi1:3.fileName = "..asmdisko19cdb-data1.vmdk"
scsi1:3.mode = "independent-persistent"
scsi1:3.redo = ""
scsi1:4.present = "TRUE"
scsi1:4.fileName = "..asmdisko19cdb-data2.vmdk"
scsi1:4.mode = "independent-persistent"
scsi1:4.redo = ""
scsi1:5.present = "TRUE"
scsi1:5.fileName = "..asmdisko19cdb-data3.vmdk"
scsi1:5.mode = "independent-persistent"
scsi1:5.redo = ""
#
# ----------------------------------------------------------------
# SHARED DISK SECTION - (END)
# ----------------------------------------------------------------
#
配置UDEV
cd /dev
for i in $(lsblk |grep disk|awk '{print $1}'|grep -v sda); do
echo "KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name",RESULT=="`/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$i`", SYMLINK+="asm-$i", OWNER="grid", GROUP="asmadmin", MODE="0660"" >>/etc/udev/rules.d/99-oracle-asmdevices.rules
done
vi /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name",RESULT=="36000c291d17055c53a62f7abcf88455a", SYMLINK+="asm-crs1", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name",RESULT=="36000c29d08a0342cdca087c34777f9f7", SYMLINK+="asm-crs2", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name",RESULT=="36000c299b5061370eef6a9d6edcfb4ee", SYMLINK+="asm-crs3", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name",RESULT=="36000c299792d931c3c73d16c6d25d7ac", SYMLINK+="asm-mgmt", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name",RESULT=="36000c295f9d043986fc840f111d477d8", SYMLINK+="asm-data", OWNER="grid", GROUP="asmadmin", MODE="0660"
# 重启生效
/sbin/partprobe /dev/sdd
/sbin/udevadm control --reload-rules
/sbin/udevadm trigger --type=devices --action=change
Verifying the Disk I/O Scheduler on Linux
# cat /sys/block/${ASM_DISK}/queue/scheduler
noop [deadline] cfq
cd /dev
for i in $(lsblk |grep disk|awk '{print $1}'|grep -v sda); do
echo $i; cat /sys/block/${i}/queue/scheduler
done
cat >> /etc/udev/rules.d/60-oracle-schedulers.rules <<EOF
ACTION=="add|change", KERNEL=="sd[b-z]", ATTR{queue/rotational}=="0", ATTR{queue/scheduler}="deadline"
EOF
/sbin/udevadm control --reload-rules
/sbin/udevadm trigger --type=devices --action=change
格式化共享磁盘（可选）
for dsk in $(ls -tr /dev/mapper/asm_*);do
dd if=/dev/zero of=${dsk} bs=1024k count=100
done
3. 软件安装
3.1 安装GI软件
安装方式的改变
- 在 12.2 中，不再需要额外的空间来解压下载的 zip 文件。GI gold image 解压到的目录就 是GRID HOME。确保使用umask 是 022 的用户 grid 进行解压
- 运行 $GRID_HOME/gridSetup.sh 开始安装，而不是之前版本的 runInstaller。
前期准备及确认
# 1. 解压软件到oracle home
su - grid
unzip -qo /ups/soft/linuxx64_12201_grid_home.zip -d /oracle/app/12.2/grid
# 2. 安装cvu包
cd /oracle/app/12.2/grid/cv/rpm
rpm -ivh cvuqdisk-1.0.10-1.rpm
scp /oracle/app/12.2/grid/cv/rpm/cvuqdisk-1.0.10-1.rpm node2:/tmp/
ssh -o stricthostkeychecking=no node2
rpm -ivh /tmp/cvuqdisk-1.0.10-1.rpm
# 3. 检查环境
cd /oracle/app/12.2/grid/
./runcluvfy.sh stage -pre crsinst -n node1,node2 -fixup -verbose > ~/grid_env_check.txt
安装配置同时不更新应用PSU补丁二进制文件
图形界面安装
# 4. 安装gi软件
su - grid
cd /oracle/app/12.2/grid/
./gridSetup.sh
默认选择Private&Asm，则为Flex 集群，若只选择为Private,则为传统的标准集群
选择YES将GIMR使用单独的DG
选择合适的磁盘及其 冗余方式
注意：在创建RAC的过程中，创建OCR/CRS磁盘组时，不用去给磁盘组定义failure group，默认即可（每个磁盘创建一个failure group）。因为normal冗余的磁盘组正常只需要两个failur group即可，但是当normal级别的磁盘组用于存放OCR时候，则必须至少包含三个 failure group。节点必须能够随时访问超过一半的votingdisk，不然则被集群逐出（脑裂)
各个节点依次顺序执行root脚本
su - root
sh /u01/app/oraInventory/orainstRoot.sh && sh /u01/app/12.2.0.1/grid/root.sh
静默安装
其中-silent指的是静默安装，-ignorePrereq忽略prerequisite的检查结果，showProgress显示进度
命令行参数方式
# 安装gi软件
su - grid
cd /oracle/app/12.2/grid/
./gridSetup.sh -ignorePrereq -skipPrereqs -waitforcompletion -silent
-responseFile ${SOFT_HOME_DIR}/install/response/gridsetup.rsp
INVENTORY_LOCATION=${ORA_INVENTORY}
SELECTED_LANGUAGES=en,en_US,zh_CN
oracle.install.option=${INSTALL_OPTION}
ORACLE_BASE=${SOFT_BASE_DIR}
oracle.install.asm.OSDBA=${OSASMDBA_GROUP}
oracle.install.asm.OSOPER=${OSASMOPER_GROUP}
oracle.install.asm.OSASM=${OSASM_GROUP}
oracle.install.crs.config.scanType=LOCAL_SCAN
oracle.install.crs.config.gpnp.scanName=${SCAN_NAME}
oracle.install.crs.config.gpnp.scanPort=${SCAN_PORT}
oracle.install.crs.config.ClusterConfiguration=${CLUSTER_CONFIGURATION}
oracle.install.crs.config.configureAsExtendedCluster=false
oracle.install.crs.config.clusterName=${CLUSTER_NAME}
oracle.install.crs.config.gpnp.configureGNS=false
oracle.install.crs.config.autoConfigureClusterNodeVIP=false
oracle.install.crs.config.clusterNodes=${CLUSTER_NODE_LIST}
oracle.install.crs.config.networkInterfaceList=${INTERFACE_LIST}
oracle.install.asm.configureGIMRDataDG=${GIMRDATADG_FLAG}
oracle.install.crs.config.storageOption=${CRS_STORAGE_TYPE}
oracle.install.crs.config.useIPMI=false
oracle.install.asm.storageOption=${ASM_STORAGE_TYPE}
oracle.install.asm.SYSASMPassword=${SYS_PWD}
oracle.install.asm.diskGroup.name=${CRS_DG_NAME}
oracle.install.asm.diskGroup.redundancy=${CRS_DG_REDUNDANCY}
oracle.install.asm.diskGroup.AUSize=${CRS_AU_SIZE}
oracle.install.asm.diskGroup.disksWithFailureGroupNames=
oracle.install.asm.diskGroup.disks=${CRS_DISK_LIST}
oracle.install.asm.diskGroup.diskDiscoveryString=${DISK_DISCOVERY_STRING}
oracle.install.asm.monitorPassword=${SYS_PWD}
oracle.install.asm.gimrDG.name=${GIRM_DG_NAME}
oracle.install.asm.gimrDG.redundancy=${GIRM_DG_REDUNDANCY}
oracle.install.asm.gimrDG.AUSize=${GIRM_AU_SIZE}
oracle.install.asm.gimrDG.disks=${GIRM_DISK_LIST}
oracle.install.asm.configureAFD=false
oracle.install.crs.configureRHPS=false
oracle.install.crs.config.ignoreDownNodes=false
oracle.install.config.managementOption=NONE
oracle.install.crs.rootconfig.executeRootScript=false
# 2) 逐个节点依次顺序执行root脚本
su - root
sh ${ORA_INVENTORY}/orainstRoot.sh && sh ${GRID_HOME}/root.sh
# 3) 创建MGMT管理资料库 【19C MGMT不是必选项】
su - grid
cd /oracle/app/12.2/grid/
GI_SETUP_RSP=$(ls -tr ${SOFT_HOME_DIR}/install/response/db_*.rsp|tail -1)
./gridSetup.sh -silent -executeConfigTools -responseFile ${GI_SETUP_RSP}
响应文件方式
准备响应文件
grep -Ev "^#|^$|[ ]+#" ${ORACLE_HOME}/install/response/gridsetup.rsp > ~/grid.rsp
# 修改响应文件内容
cat > ~/gridsetup.rsp<<-EOF
oracle.install.responseFileVersion=/oracle/install/rspfmt_crsinstall_response_schema_v19.0.0
oracle.install.option=CRS_CONFIG
oracle.install.asm.OSDBA=asmdba
oracle.install.asm.OSOPER=asmoper
oracle.install.asm.OSASM=asmadmin
oracle.install.crs.config.scanType=LOCAL_SCAN
oracle.install.crs.config.SCANClientDataFile=
oracle.install.crs.config.gpnp.scanName=o19c-scan
oracle.install.crs.config.gpnp.scanPort=1533
oracle.install.crs.config.ClusterConfiguration=STANDALONE
oracle.install.crs.config.configureAsExtendedCluster=false
oracle.install.crs.config.memberClusterManifestFile=
oracle.install.crs.config.clusterName=o19c-cluster
oracle.install.crs.config.gpnp.configureGNS=false
oracle.install.crs.config.autoConfigureClusterNodeVIP=false
oracle.install.crs.config.gpnp.gnsOption=
oracle.install.crs.config.gpnp.gnsClientDataFile=
oracle.install.crs.config.gpnp.gnsSubDomain=
oracle.install.crs.config.gpnp.gnsVIPAddress=
oracle.install.crs.config.sites=
oracle.install.crs.config.clusterNodes=o19c1:o19c1-vip:HUB,o19c2:o19c2-vip:HUB
oracle.install.crs.config.networkInterfaceList=ens32:192.168.10.0:1,ens33:172.168.0.0:5
oracle.install.crs.configureGIMR=false
oracle.install.asm.configureGIMRDataDG=false
oracle.install.crs.config.storageOption=
oracle.install.crs.config.sharedFileSystemStorage.votingDiskLocations=
oracle.install.crs.config.sharedFileSystemStorage.ocrLocations=
oracle.install.crs.config.useIPMI=false
oracle.install.crs.config.ipmi.bmcUsername=
oracle.install.crs.config.ipmi.bmcPassword=
oracle.install.asm.SYSASMPassword=oracle
oracle.install.asm.diskGroup.name=CRSDG
oracle.install.asm.diskGroup.redundancy=NORMAL
oracle.install.asm.diskGroup.AUSize=4
oracle.install.asm.diskGroup.FailureGroups=
oracle.install.asm.diskGroup.disksWithFailureGroupNames=
oracle.install.asm.diskGroup.disks=/dev/asm-crs1,/dev/asm-crs2,/dev/asm-crs3
oracle.install.asm.diskGroup.quorumFailureGroupNames=
oracle.install.asm.diskGroup.diskDiscoveryString=/dev/asm-*
oracle.install.asm.monitorPassword=oracle
oracle.install.asm.gimrDG.name=MGMT
oracle.install.asm.gimrDG.redundancy=EXTERNAL
oracle.install.asm.gimrDG.AUSize=4
oracle.install.asm.gimrDG.FailureGroups=
oracle.install.asm.gimrDG.disksWithFailureGroupNames=
oracle.install.asm.gimrDG.disks=/dev/asm-mgmt
oracle.install.asm.gimrDG.quorumFailureGroupNames=
oracle.install.asm.configureAFD=false
oracle.install.crs.configureRHPS=false
oracle.install.crs.config.ignoreDownNodes=false
oracle.install.config.managementOption=NONE
oracle.install.config.omsHost=
oracle.install.config.omsPort=
oracle.install.config.emAdminUser=
oracle.install.config.emAdminPassword=
oracle.install.crs.rootconfig.executeRootScript=false
oracle.install.crs.rootconfig.configMethod=
oracle.install.crs.rootconfig.sudoPath=
oracle.install.crs.rootconfig.sudoUserName=
oracle.install.crs.config.batchinfo=
oracle.install.crs.app.applicationAddress=
oracle.install.crs.deleteNode.nodes=
EOF
# 执行命令
${ORACLE_HOME}/gridSetup.sh -silent -ignorePrereq -skipPrereqs -waitforcompletion -responseFile ~/gridsetup.rsp
GI软件配置时升级PSU补丁文件
[参考文档](How to Apply a Grid Infrastructure Patch Before Grid Infrastructure Configuration (before root.sh or rootupgrade.sh or gridsetup.bat) is Executed (Doc ID 1410202.1))
必须在未执行 root.sh or rootupgrade.sh or gridsetup.bat脚本前
这种方式先应用GI PSU二进制文件然后进行GI集群配置
19.7版本更新（RU）包含许多与在OL8/RHEL8上运行Oracle 19c RAC有关的修补程序，建议新部署环境时使用-applyRU选项安装部署
解压基础软件包到GI Home目录并更新OPatch
# 1. 解压软件到gi home (grid用户)
su - grid
unzip -qo /ups/soft/linuxx64_12201_grid_home.zip -d /oracle/app/12.2/grid
# 2. 更新opatch (grid用户)
unzip -d /oracle/app/12.2/grid p6880880_<platform version info>.zip
实际案例
/ups/oracle/o19c/grid/gridSetup.sh -silent -ignorePrereq -skipPrereqs -waitforcompletion -responseFile /ups/soft/gridsetup.rsp -applyRU /ups/soft/31750108
安装/更新补丁使用方法
用法
Usage: gridSetup.sh [<flag>] [<option>]
Following are the possible flags:
-help - display help.
-silent - run in silent mode. The inputs can be a response file or a list of command line variable value pairs.
[-ignorePrereqFailure - ignore all prerequisite checks failures.]
[-lenientInstallMode - perform the best effort installation by automatically ignoring invalid data in input parameters.]
-responseFile - specify the complete path of the response file to use.
-logLevel - enable the log of messages up to the priority level provided in this argument. Valid options are: severe, warning, info, config, fine, finer, finest.
-executePrereqs | -executeConfigTools | -createGoldImage | -switchGridHome | -downgrade | -dryRunForUpgrade
-executePrereqs - execute the prerequisite checks only.
-executeConfigTools - execute the config tools for an installed home.
[-skipStackCheck - skip the stack status check.]
-createGoldImage - create a gold image from the current Oracle home.
-destinationLocation - specify the complete path to where the created gold image will be located.
[-exclFiles - specify the complete paths to the files to be excluded from the new gold image.]
-switchGridHome - change the Oracle Grid Infrastructure home path.
-downgrade - To downgrade Grid Infrastructure back to old home (to be used only in the case of incomplete upgrade).
-silent - run in silent mode. The inputs can be a response file or a list of command line variable value pairs.
[-ignorePrereqFailure - ignore all prerequisite checks failures.]
[-lenientInstallMode - perform the best effort installation by automatically ignoring invalid data in input parameters.]
[-configmethod - Specify the method to execute scripts as privileged user. If not specified then user will be instructed to run the scripts by logging in as privileged user. Valid options are: root,sudo.]
[-sudopath - Specify the complete path to the sudo program. This is an optional argument. This is needed if 'sudo' is specified for the configmethod and 'sudo' program is not present in the default path.]
[-sudousername - Specify the name of sudoer.]
-dryRunForUpgrade - To perform a dry run of the Grid Infrastructure Upgrade process.
-debug - run in debug mode.
-printdiskusage - log the debug information for the disk usage.
-printmemory - log the debug information for the memory usage.
-printtime - log the debug information for the time usage.
-waitForCompletion - wait for the completion of the installation, instead of spawning the installer and returning the console prompt.
-noconfig - do not execute the config tools.
-noconsole - suppress the display of messages in the console. The console is not allocated.
-ignoreInternalDriverError - ignore any internal driver errors.
-noCopy - perform the configuration without copying the software on to the remote nodes.
-applyRU - apply release update to the Oracle home.
-applyOneOffs - apply one-off patch to the Oracle home. Multiple one-off patches can be passed as a comma separated list of locations.
12.2.0.1
# 3. 应用补丁 (grid用户)
# 解压补丁文件
unzip -qo p27468969*.zip -d /oracle/app/12.2/grid/
# 3.1 应用RU补丁
$GI_HOME/gridSetup.sh -applyPSU <downloaded patch location>
# 3.2 应用 Non-RU 补丁
$GI_HOME/gridSetup.sh -applyOneOffs <downloaded patch location>
# 3.3 应用 RU 和 Non-RU 补丁
$GI_HOME/gridSetup.sh -applyPSU <downloaded patch location> -applyOneOffs <downloaded patch location>
18.1 and above
To apply only Release Updates:
$GI_HOME/gridSetup.sh -silent -ignorePrereq -skipPrereqs -waitforcompletion -applyRU <downloaded patch location> -responseFile /ups/soft/gridsetup.rsp
For Windows: $GI_HOME/gridSetup.bat -silent -applyRU <downloaded patch location>
To apply only Non-RU patches:
$GI_HOME/gridSetup.sh -silent -ignorePrereq -skipPrereqs -waitforcompletion -applyRUR <downloaded patch location> -responseFile /ups/soft/gridsetup.rsp
For Windows: $GI_HOME/gridSetup.bat -silent -applyOneOffs <downloaded patch location>
To apply Release Updates and Non-RU patches in one command:
$GI_HOME/gridSetup.sh -silent -ignorePrereq -skipPrereqs -waitforcompletion -applyRU <downloaded patch location> -applyRUR <downloaded atch location> -responseFile /ups/soft/gridsetup.rsp
检查集群服务状态
su - grid
crsctl status res -t -init
crsctl status res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr
ONLINE ONLINE ol6-122-rac1 STABLE
ONLINE ONLINE ol6-122-rac2 STABLE
ora.CRS.dg
ONLINE ONLINE ol6-122-rac1 STABLE
ONLINE ONLINE ol6-122-rac2 STABLE
ora.MGMT.dg
ONLINE ONLINE ol6-122-rac1 STABLE
ONLINE ONLINE ol6-122-rac2 STABLE
ora.LISTENER.lsnr
ONLINE ONLINE ol6-122-rac1 STABLE
ONLINE ONLINE ol6-122-rac2 STABLE
ora.chad
ONLINE ONLINE ol6-122-rac1 STABLE
ONLINE ONLINE ol6-122-rac2 STABLE
ora.net1.network
ONLINE ONLINE ol6-122-rac1 STABLE
ONLINE ONLINE ol6-122-rac2 STABLE
ora.ons
ONLINE ONLINE ol6-122-rac1 STABLE
ONLINE ONLINE ol6-122-rac2 STABLE
ora.proxy_advm
OFFLINE OFFLINE ol6-122-rac1 STABLE
OFFLINE OFFLINE ol6-122-rac2 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE ol6-122-rac2 STABLE
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE ol6-122-rac1 STABLE
ora.LISTENER_SCAN3.lsnr
1 ONLINE ONLINE ol6-122-rac1 STABLE
ora.MGMTLSNR
1 ONLINE ONLINE ol6-122-rac1 169.254.15.226 192.1
68.1.201,STABLE
ora.asm
1 ONLINE ONLINE ol6-122-rac1 Started,STABLE
2 ONLINE ONLINE ol6-122-rac2 Started,STABLE
3 OFFLINE OFFLINE STABLE
ora.cvu
1 ONLINE ONLINE ol6-122-rac1 STABLE
ora.mgmtdb
1 ONLINE ONLINE ol6-122-rac1 Open,STABLE
ora.ol6-122-rac1.vip
1 ONLINE ONLINE ol6-122-rac1 STABLE
ora.ol6-122-rac2.vip
1 ONLINE ONLINE ol6-122-rac2 STABLE
ora.qosmserver
1 ONLINE ONLINE ol6-122-rac1 STABLE
ora.scan1.vip
1 ONLINE ONLINE ol6-122-rac2 STABLE
ora.scan2.vip
1 ONLINE ONLINE ol6-122-rac1 STABLE
ora.scan3.vip
1 ONLINE ONLINE ol6-122-rac1 STABLE
--------------------------------------------------------------------------------
$
3.2 创建ASM DG
图形界面创建DG
su - grid
asmca
依次按需创建ASM DG
完成后退出。
asmca -silent命令方式创建DG
# 创建外部冗余磁盘组 external
asmca -silent -createDiskGroup -diskString '/dev/asm-*' -diskGroupName DATA2 -disk '/dev/asm-data1' -redundancy EXTERNAL -au_size 4
asmca -silent -createDiskGroup -diskString '/dev/asm-*' -diskGroupName DATA3 -diskList '/dev/asm-data2','/dev/asm-data3' -redundancy EXTERNAL -au_size 4 # -compatible.asm 12.2 -compatible.rdbms 12.2 -compatible.advm 12.2
# 创建Normal磁盘组
asmca -silent -createDiskGroup -diskString '/dev/asm-*' -diskGroupName DATA4 -diskList '/dev/asm-data4','/dev/asm-data5' -redundancy NORMAL -au_size 4
3.3 安装DB软件
解压软件包
su - oracle
unzip -qo /ups/soft/linuxx64_12201_database.zip -d ${ORACLE_HOME}
建议执行 runInstaller 命令安装期间应用补丁
使用-applyRU 或者 -applyOneOffs 选项
如果是 19.7 RU 或者更高版本
===========================================
$ export CV_ASSUME_DISTID=OL7
$ ./runInstaller -applyRU <19.7DBRU patch 30869156 unzip location>
比如-
$ ./runInstaller -applyRU /u01/app/30869156
这里的-
30869156 - DATABASE RELEASE UPDATE 19.7.0.0.0
如果是 19.6 RU
===========================================
$ cd /u01/app/oracle/product/19c/dbhome_1
$ export CV_ASSUME_DISTID=OL7
$ ./runInstaller -applyRU <19.6 DBRU patch 30557433 unzip location> -applyOneOffs <19.6 OJVM patch 30484981 unzip location>,<19.6 Oneoff patch 30150710 unzip location>
比如-
$ ./runInstaller -applyRU /u01/app/30557433 -applyOneOffs /u01/app/30484981,/u01/app/30150710
这里的-
30557433 - DATABASE RELEASE UPDATE 19.6.0.0.0
30484981 - OJVM RELEASE UPDATE 19.6.0.0.0
30150710 - EM express OneOff patch 19.6.0.0.0
注意-
1. 在运行 Installer 之前下载最新的 OPatch
2. 在这两种情况下，安装RU后，OCW RU仍然是19.3.0.0.0.0版本。终端用户可以选择在安装后或安装过程中更新OCW版本(参见下面的步骤a和b)。
a. 如果是 19.7 RU 或者更高版本
===========================================
$ export CV_ASSUME_DISTID=OL7
$ ./runInstaller -applyRU <19.7 DBRU patch 30869156 unzip location> -applyOneOffs <19.7 OCWRU patch 30894985 unzip location>
比如-
$ ./runInstaller -applyRU /u01/app/30869156 -applyOneOffs /u01/app/30899722/30894985
这里的-
30869156 - DATABASE RELEASE UPDATE 19.7.0.0.0
30894985 - OCWRU Patch 19.7.0.0.0 (Not separately available for download.Download GI RU 19.7.0.0.0 Patch 30899722 & then unzip to locate the OCWRU Patch)
b. 如果是 19.6 RU
===========================================
$ export CV_ASSUME_DISTID=OL7
$ ./runInstaller -applyRU <19.6DBRU patch 30557433 unzip location> -applyOneOffs <19.6 OJVM patch 30484981 unzip location>,<19.6 Oneoff patch 30150710 unzip location>,<19.6 OCWRU patch 30489227 unzip location>
比如-
$ ./runInstaller -applyRU /u01/app/30557433 -applyOneOffs /u01/app/30484981,/u01/app/30150710,/u01/app/30501910/30489227
这里的-
30557433 - DATABASE RELEASE UPDATE 19.6.0.0.0
30484981 - OJVM RELEASE UPDATE 19.6.0.0.0
30150710 - EM express OneOff patch 19.6.0.0.0
30489227 - OCWRU Patch 19.6.0.0.0 (Not separately available for download.Download GI RU 19.6.0.0.0 Patch 30501910 & then unzip to locate the OCWRU Patch)
3. EM Express One-Off Patch 30150710 已包含在 19.7 DBRU Patch 30869156
图形界面安装
su - oracle
cd ${ORACLE_HOME}
./runInstaller
依次逐个节点执行root脚本
su - root
sh /u01/app/oracle/product/12.2.0.1/db_1/root.sh
静默安装
使用说明
# 1. 使用说明
${ORACLE_HOME}/runInstaller -silent -h
Usage: runInstaller [<flag>] [<option>]
Following are the possible flags:
-help - display help.
-silent - run in silent mode. The inputs can be a response file or a list of command line variable value pairs.
[-ignorePrereqFailure - ignore all prerequisite checks failures.]
-responseFile - specify the complete path of the response file to use.
-logLevel - enable the log of messages up to the priority level provided in this argument. Valid options are: severe, warning, info, config, fine, finer, finest.
-executePrereqs | -executeConfigTools | -createGoldImage
-executePrereqs - execute the prerequisite checks only.
-executeConfigTools - execute the config tools for an installed home.
-createGoldImage - create a gold image from the current Oracle home.
-destinationLocation - specify the complete path to where the created gold image will be located.
[-exclFiles - specify the complete paths to the files to be excluded from the new gold image.]
-debug - run in debug mode.
-printdiskusage - log the debug information for the disk usage.
-printmemory - log the debug information for the memory usage.
-printtime - log the debug information for the time usage.
-waitForCompletion - wait for the completion of the installation, instead of spawning the installer and returning the console prompt.
-noconfig - do not execute the config tools.
-noconsole - suppress the display of messages in the console. The console is not allocated.
-ignoreInternalDriverError - ignore any internal driver errors.
-noCopy - perform the configuration without copying the software on to the remote nodes.
-applyRU - apply release update to the Oracle home.
-applyOneOffs - apply one-off patch to the Oracle home. Multiple one-off patches can be passed as a comma separated list of locations.
软件安装配置
响应文件方式
# 修改配置影响文件方式
cat > /ups/soft/db_install.rsp <<-EOF
oracle.install.responseFileVersion=/oracle/install/rspfmt_dbinstall_response_schema_v19.0.0
oracle.install.option=INSTALL_DB_SWONLY
oracle.install.db.InstallEdition=EE
oracle.install.db.OSDBA_GROUP=dba
oracle.install.db.OSOPER_GROUP=oper
oracle.install.db.OSBACKUPDBA_GROUP=backupdba
oracle.install.db.OSDGDBA_GROUP=dgdba
oracle.install.db.OSKMDBA_GROUP=kmdba
oracle.install.db.OSRACDBA_GROUP=racdba
oracle.install.db.rootconfig.executeRootScript=false
oracle.install.db.rootconfig.configMethod=
oracle.install.db.rootconfig.sudoPath=
oracle.install.db.rootconfig.sudoUserName=
oracle.install.db.CLUSTER_NODES=o19c1,o19c2
oracle.install.db.config.starterdb.type=GENERAL_PURPOSE
oracle.install.db.config.starterdb.globalDBName=o19cdb
oracle.install.db.config.starterdb.SID=o19cdb
oracle.install.db.ConfigureAsContainerDB=true
oracle.install.db.config.PDBName=pdb_orcl
oracle.install.db.config.starterdb.characterSet=AL32UTF8
oracle.install.db.config.starterdb.memoryOption=false
oracle.install.db.config.starterdb.memoryLimit=2048
oracle.install.db.config.starterdb.installExampleSchemas=false
oracle.install.db.config.starterdb.password.ALL=oracle
oracle.install.db.config.starterdb.password.SYS=
oracle.install.db.config.starterdb.password.SYSTEM=
oracle.install.db.config.starterdb.password.DBSNMP=
oracle.install.db.config.starterdb.password.PDBADMIN=oracle
oracle.install.db.config.starterdb.managementOption=
oracle.install.db.config.starterdb.omsHost=
oracle.install.db.config.starterdb.omsPort=
oracle.install.db.config.starterdb.emAdminUser=
oracle.install.db.config.starterdb.emAdminPassword=
oracle.install.db.config.starterdb.enableRecovery=false
oracle.install.db.config.starterdb.storageType=ASM_STORAGE
oracle.install.db.config.starterdb.fileSystemStorage.dataLocation=
oracle.install.db.config.starterdb.fileSystemStorage.recoveryLocation=
oracle.install.db.config.asm.diskGroup=DATA
oracle.install.db.config.asm.ASMSNMPPassword=oracle
EOF
${ORACLE_HOME}/runInstaller -silent -waitForCompletion -ignorePrereqFailure -responseFile /ups/soft/db_install.rsp -ignoreSysPrereqs
命令行方式
su - oracle
cd /ups/soft/database
# 执行./runInstaller启用界面 【命令行参数】
./runInstaller -ignorePrereq -waitforcompletion -silent
-responseFile ${DB_HOME_DIR}/install/response/db_install.rsp
oracle.install.option=${INSTALL_OPTION}
ORACLE_HOSTNAME=${ORA_HOST_NAME}
UNIX_GROUP_NAME=${UNIX_GROUP_NAME}
INVENTORY_LOCATION=${ORA_INVENTORY}
SELECTED_LANGUAGES=en,en_US,zh_CN
ORACLE_HOME=${DB_HOME_DIR}
ORACLE_BASE=${DB_BASE_DIR}
oracle.install.db.InstallEdition=${INSTALL_EDITION}
oracle.install.db.OSDBA_GROUP=${OSDBA_GROUP}
oracle.install.db.OSOPER_GROUP=${OSOPER_GROUP}
oracle.install.db.OSBACKUPDBA_GROUP=${OSBACKUPDBA_GROUP}
oracle.install.db.OSDGDBA_GROUP=${OSDGDBA_GROUP}
oracle.install.db.OSKMDBA_GROUP=${OSKMDBA_GROUP}
oracle.install.db.OSRACDBA_GROUP=${OSRACDBA_GROUP}
SECURITY_UPDATES_VIA_MYORACLESUPPORT=false
DECLINE_SECURITY_UPDATES=true ${GI_PARAMS}
# 2) 执行root脚本
su - root
sh ${ORA_INVENTORY}/orainstRoot.sh && sh ${DB_HOME_DIR}/root.sh
3.4 建库前安装软件补丁（可选）
使用root用户执行补丁更新命令，GI打补丁需要用GI $ORACLE_HOME/OPatch/opatchauto，DB打补丁需要使用 DB $ORACLE_HOME/OPatch/opatchauto
su - root
/ups/app/19c/grid/OPatch/opatchauto apply /oracle/soft/<patchid> -oh /ups/app/19c/grid
# 由于没有创建实例，OCR中没有DB HOME信息，从而不能更新DB HOME，需要单独指定目录更新补丁
su - root
/ups/app/oracle/database/19c/db_1/OPatch/opatchauto apply /oracle/soft/<patchid> -oh /ups/app/oracle/database/19c/db_1
3.5 DBCA工具配置数据库
图形界面建库
su - oracle
dbca
根据需要是否创建PDB
这里都全选，包含PDB。避免后续业务需要组件需要在PDB上另外单独安装配置。
根据实际配置SGA+PGA
配置数据块大小和进程数
建议：12C+版本选择默认的字符集和国际字符集配置（即：AL32UTF8和AL32UTF16），业务根据需要单独配置PDB字符集。
静默方式建库
# Data_Warehouse.dbc|General_Purpose.dbc|New_Database.dbt
TEMPLATE_NAME='General_Purpose.dbc'
# SINGLE | RAC | RACONENODE
DB_CONING_TYPE='RAC'
DB_NAME='orcl'
# AL32UTF8 | ZHS16GBK
CHARACTERSET='AL32UTF8'
SYS_PWD='oracle'
# MULTIPURPOSE | OLTP
DB_TYPE='OLTP'
# ASM |FS
STORAGE_TYPE='ASM'
MEMORY_PCT='40'
# 数据文件目录
DATA_DIR='DATA'
USE_OMF='true'
ISCDB='true'
INITPARAMS='processes=1000'
NODE_LIST='node1,node2'
# su - oracle
dbca -silent -ignorePreReqs -createDatabase
-databaseConfigType ${DB_CONING_TYPE}
-datafileJarLocation ${ORACLE_HOME}/assistants/dbca/templates/
-templateName ${TEMPLATE_NAME}
-gdbname ${DB_NAME} -sid ${DB_NAME} -responseFile NO_VALUE
-characterSet ${CHARACTERSET}
-sysPassword "${SYS_PWD}"
-systemPassword "${SYS_PWD}"
-databaseType ${DB_TYPE}
-automaticMemoryManagement false
-memoryPercentage ${MEMORY_PCT}
-storageType ${STORAGE_TYPE}
-datafileDestination '"+${DATA_DIR}"'
-recoveryAreaDestination NONE
-emConfiguration NONE
-initParams db_create_file_dest='"+${DATA_DIR}"'
-initParams ${INITPARAMS}
-createAsContainerDatabase ${ISCDB}
-nodelist ${NODE_LIST} -asmsnmpPassword "${SYS_PWD}"
-useOMF ${USE_OMF} ${LISTENER_PARAMS}
# 在 DBCA 静默安装模式下可以指定 control_file 初始化参数来修改 control file 的路径：
-initparams control_files='+DATA'
比如:
dbca -silent -ignorePreReqs -createDatabase -databaseConfigType RAC -datafileJarLocation /ups/oracle/database/product/12.2/db_1/assistants/dbca/templates/ -templateName General_Purpose.dbc -gdbname o12cdb -sid o12cdb -responseFile NO_VALUE -characterSet AL32UTF8 -sysPassword oracle -systemPassword oracle -databaseType OLTP -automaticMemoryManagement false -memoryPercentage 40 -storageType ASM -datafileDestination '+DATA' -recoveryAreaDestination NONE -emConfiguration NONE -initParams db_create_file_dest='+DATA' -initParams processes=1000 -initparams control_files='+CTL1','+CTL2' -createAsContainerDatabase true -nodelist orc1,orc2 -asmsnmpPassword oracle -useOMF true
响应文件
dbca -createDatabase -silent -ignorePreReqs -ignorePrereqFailure -responseFile /ups/soft/dbca.rsp
检查服务状态
srvctl config database -d orcl
srvctl status database -d orcl
. /home/grid/.bash_profile
crsctl status res -t
删除数据库
${ORACLE_HOME}/bin/dbca -silent -ignorePreReqs -ignorePrereqFailure -deleteDatabase -sourceDB o19c -sysDBAUserName sys -sysDBAPassword oracle
添加实例
# 添加实例
${ORACLE_HOME}/bin/dbca -silent -addInstance -gdbName o19c -nodeName o19c2 -instanceName o19c2 -sysDBAUserName sys -sysDBAPassword oracle
5. 数据库配置调整
5.1 调整控制文件
Oracle使用ASM存储，建库时默认只有一个控制文件，需手动添加2个控制文件。安装数据库默认设置control_files为3个，==分别存放不同VG/ASMDG，即创建3个VG/ASMDG。==
-- 1. 查询当前控制文件名称信息
sqlplus "/ as sysdba"
SQL> select value from v$parameter where name='control_files';
-- 2. 备份参数文件
SQL> set verify off
SQL> column timecol new_value timestamp noprint
SQL> SELECT to_char(sysdate,'yyyymmddhh24') timecol FROM dual;
SQL> create pfile='/tmp/pfile_&×tamp..ora' from spfile;
-- 3. 添加spfile中的控制文件
-- 命令格式： alter system set control_files='[步骤1中一个控制文件名称路径]','[新增文件路径]'... scope=spfile sid='*';
-- 例子：
SQL> alter system set control_files='+DATA/ORCL/CONTROLFILE/current.279.1033659949','+CTL1','+CTL2','+CTL3' scope=spfile sid='*';
SQL> quit;
# 4. 将数据库实例重启动到nomount状态
su - oracle
$ srvctl stop database -d orcl
# 其中一个实例进行变更实施
$ srvctl start instance -d orcl -i orcl1 -o nomount
# 5. 使用rman还原controlfile
$ rman target /
RMAN> restore controlfile from '+DATA/ORCL/CONTROLFILE/current.279.1033659949';
Starting restore at 19-MAY-15
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=1153 device type=DISK
channel ORA_DISK_1: copied control file copy
output file name=+CTL1/ORCL/CONTROLFILE/current.267.880113159
output file name=+CTL2/ORCL/CONTROLFILE/current.269.880122971
output file name=+CTL3/ORCL/CONTROLFILE/current.270.880122971
Finished restore at 19-MAY-15
RMAN> sql 'alter database mount';
sql statement: alter database mount
released channel: ORA_DISK_1
RMAN> sql 'alter database open';
sql statement: alter database open
RMAN> exit
# 6. 将第5步创建的控制文件更新到spfile文件
sqlplus "/ as sysdba"
SQL> alter system set control_files='+CTL1/ORCL/CONTROLFILE/current.267.880113159' ,'+CTL2/ORCL/CONTROLFILE/current.269.880122971', '+CTL3/ORCL/CONTROLFILE/current.270.880122971' scope=spfile sid='*';
-- 再次备份spfile文件(可选)
SQL> create pfile='/tmp/pfile1.ora' from spfile;
SQL> quit;
# 7. 关闭DB实例1
$ srvctl stop instance -d orcl -i orcl1
# 8. 启动数据库服务
$ srvctl start instance -d orcl -i orcl1
$ srvctl start instance -d orcl -i orcl2
# 9. 检查确认
sqlplus "/ as sysdba"
SQL> select value from v$parameter where name='control_files';
SQL> show parameter control_file
su - oracle
$ . /home/grid/.bash_profile
$ crsctl status res -t
5.2 调整Redo文件配置
无特殊要求创建6组redo，大小1G，2个成员分别存放==不同VG/ASM DG==上，即创建2个VG/ASM DG。注意后缀不用log，以免误删。
sqlplus "/ as sysdba"
-- 1. 查询当前redo 日志文件
set lines 168
col member for a61
col INSTANCE_NAME for a13
col status for a10
col archived for a8
SELECT
i.instance_name
, i.thread#
, f.group#
, f.member
, f.type
, l.status
, l.bytes/1048576 size_mb
, l.archived
FROM
gv$logfile f
, gv$log l
, gv$instance i
WHERE
f.group# = l.group#
AND l.thread# = i.thread#
AND i.inst_id = f.inst_id
AND f.inst_id = l.inst_id
ORDER BY
i.instance_name
, f.group#
, f.member;
-- 2. 添加日志组，每个日志2个成员，日志大小1G
-- 实例1
ALTER DATABASE ADD LOGFILE THREAD 1 GROUP 11 ('+REDO1','+REDO2') size 1024m;
ALTER DATABASE ADD LOGFILE THREAD 1 GROUP 12 ('+REDO1','+REDO2') size 1024m;
ALTER DATABASE ADD LOGFILE THREAD 1 GROUP 13 ('+REDO1','+REDO2') size 1024m;
ALTER DATABASE ADD LOGFILE THREAD 1 GROUP 14 ('+REDO1','+REDO2') size 1024m;
ALTER DATABASE ADD LOGFILE THREAD 1 GROUP 15 ('+REDO1','+REDO2') size 1024m;
ALTER DATABASE ADD LOGFILE THREAD 1 GROUP 16 ('+REDO1','+REDO2') size 1024m;
-- 实例2
ALTER DATABASE ADD LOGFILE THREAD 2 GROUP 21 ('+REDO1','+REDO2') size 1024m;
ALTER DATABASE ADD LOGFILE THREAD 2 GROUP 22 ('+REDO1','+REDO2') size 1024m;
ALTER DATABASE ADD LOGFILE THREAD 2 GROUP 23 ('+REDO1','+REDO2') size 1024m;
ALTER DATABASE ADD LOGFILE THREAD 2 GROUP 24 ('+REDO1','+REDO2') size 1024m;
ALTER DATABASE ADD LOGFILE THREAD 2 GROUP 25 ('+REDO1','+REDO2') size 1024m;
ALTER DATABASE ADD LOGFILE THREAD 2 GROUP 26 ('+REDO1','+REDO2') size 1024m;
-- 3. 切换日志，确认日志组状态为'INACTIVE'，既可以删除
alter system switch logfile;
alter system checkpoint;
-- 4. 依次删除要废弃的日志组（可选）
ALTER DATABASE drop LOGFILE GROUP 1;
ALTER DATABASE drop LOGFILE GROUP 2;
......
-- 5. 再次检查确认
set lines 168
col member for a61
col INSTANCE_NAME for a13
col status for a10
col archived for a8
SELECT
i.instance_name
, i.thread#
, f.group#
, f.member
, f.type
, l.status
, l.bytes/1048576 size_mb
, l.archived
FROM
gv$logfile f
, gv$log l
, gv$instance i
WHERE
f.group# = l.group#
AND l.thread# = i.thread#
AND i.inst_id = f.inst_id
AND f.inst_id = l.inst_id
ORDER BY
i.instance_name
, f.group#
, f.member;
5.3 调整进程数(可选)
-- 备份参数文件
SQL> set verify off
SQL> column timecol new_value timestamp noprint
SQL> SELECT to_char(sysdate,'yyyymmddhh24') timecol FROM dual;
SQL> create pfile='/tmp/pfile_&×tamp..ora' from spfile;
-- 调整参数，process需要重启实例生效
SQL> alter system set processes=3000 scope=spfile sid='*';
SQL> alter system set open_cursors=3000 scope=both sid='*';
-- 调整后确认
select name,
value,
p.ISDEFAULT,
p.ISSES_MODIFIABLE,
p.ISSYS_MODIFIABLE,
p.ISINSTANCE_MODIFIABLE,
p.DESCRIPTION
from v$parameter p
where name in ('processes', 'open_cursors');
------------------------------------------------------------
-- 参数说明：
Processes - user processes
open_cursors - max # cursors per session
5.4 数据库集群内存配置
OS HugePage特性跟数据库AMM冲突的。
- 禁用OS 大页时，数据库使用Automatic Memory Management（AMM）内存机制
- 启用OS大页时，数据库使用Automatic Shared Memory Management（ASMM）内存机制
Automatic Memory Management (AMM) on 11g & 12c (Doc ID 443746.1)
5.4.1 DB 实例内存配置(可选)
[hugepages_settings.sh计算脚本](Doc ID 401749.1)
配置HugePages(禁用AMM)
- 设置memlock(添加memlock的限制，注意该值略微小于实际物理内存的大小)
vi /etc/security/limits.conf
* soft memlock 60397977
* hard memlock 60397977
- DB实例中禁用AMM
-- 设置初始化参数MEMORY_TARGET 和MEMORY_MAX_TARGET 为0
- 计算vm.nr_hugepages的建议值
sh ./hugepages_settings.sh
- 设置vm.nr_hugepages参数
vi /etc/sysctl.d/97-oracle.conf
vm.nr_hugepages = 1496
-
停止所有实例，并重启服务器
-
验证配置
# 确保所有的数据库实例都已经启动后，检查HugePage的状态
grep HugePages /proc/meminfo
# 确保HugePages配置的有效性，HugePages_Free值应该小于HugePages_Total 的值，并且应该等于HugePages_Rsvd的值。Hugepages_Free 和HugePages_Rsvd 的值应该小于SGA 分配的gages。
5.4.2 ASM 实例内存配置（可选）
默认情况下ASM instance 也是使用AMM的，但因为ASM 实例不需要大SGA，所以对ASM 实例使用HugePages意义不大。
实例需要pool池需要的容量随着ASM DG的容量增加而增加，为了避免出现ORA-4031错误，根据实际情况按需调整ASM 实例内存。
-- 1. AMM： 建议ASM的版本低于12.1，按以下配置
SQL> alter system set memory_max_target=4096m scope=spfile;
SQL> alter system set memory_target=1536m scope=spfile;
--备份参数文件
su - grid
sqlplus / as sysdba
create pfile='/home/grid/pfile.ora' from spfile;
SQL> alter system set memory_max_target=4096m scope=spfile;
SQL> alter system set memory_target=1536m scope=spfile;
--检查确认
sqlplus / as sysdba
show parameter target
quit;
-- 2. ASMM： 查询ASM存储容量及冗余，后计算SHARED_POOL_SIZE值
SELECT SUM(bytes)/(1024*1024*1024) FROM V$DATAFILE;
SELECT SUM(bytes)/(1024*1024*1024) FROM V$LOGFILE a, V$LOG b
WHERE a.group#=b.group#;
SELECT SUM(bytes)/(1024*1024*1024) FROM V$TEMPFILE
WHERE status='ONLINE';
- For disk groups using external redundancy, every 100 GB of space needs 1 MB of extra shared pool plus 2 MB.
- For disk groups using normal redundancy, every 50 GB of space needs 1 MB of extra shared pool plus 4 MB.
- For disk groups using high redundancy, every 33 GB of space needs 1 MB of extra shared pool plus 6 MB.
-- 3. 在11.2.0.3/11.2.0.4
1) If PROCESSES parameter is explicitly set:
The MEMORY_TARGET should be set to no less than:
256M + PROCESSES * 132K (64bit)
256M + PROCESSES * 120K (32bit)
2) If PROCESSES parameter is not set:
The MEMORY_TARGET should be set to no less than:
256M + (available_cpu_cores * 80 + 40) * 132K (64bit)
256M + (available_cpu_cores * 80 + 40) * 120K (32bit)
5.5 配置归档模式（可选）
-- 1. 备份参数文件
create pfile='/home/oracle/pfile2.ora' from spfile;
-- 2. 配置归档目录
alter system set log_archive_dest_1='location=+ARCH' scope=spfile sid='*';
-- 3. 关闭数据库
$ srvctl stop instance -d orcl -i orcl1
$ srvctl stop instance -d orcl -i orcl2
-- 4. 启动一个实例到mount状态
$ srvctl start instance -d orcl -i orcl1 -o mount
-- 5. 修改数据库配置
sqlplus / as sysdba
alter database archivelog;
-- 6. 重启数据库
$ srvctl stop instance -d orcl -i orcl1
$ srvctl start instance -d orcl -i orcl1
$ srvctl start instance -d orcl -i orcl2
-- 7. 检查确认
archive log list
注意归档目录空间增长
# 定期清理脚本
#!/bin/sh
export LANG=C
ECHO="echo"
export OSTYPE=$(uname -s)
case ${OSTYPE} in
"HP-UX")
;;
"AIX")
;;
"SunOS")
;;
*) ECHO="echo -e"
esac
# 加载 profile
PROFILE="$1"
[[ -f ${PROFILE} ]] && . ${PROFILE}
# 实例名称
INST_NAME="$2"
[[ ! -z "${INST_NAME}" ]] && export ORACLE_SID=${INST_NAME}
[[ -z "${LOG_DIR}" ]] && LOG_DIR='/tmp'
[[ -z "${SQLPLUS_CONN}" ]] && SQLPLUS_CONN='/ as sysdba'
[[ -z "${SQLPLUS_CONN}" ]] && RMAN_CONN='/'
# 日志保留期限
KEEP_TIME_HOURS="$3"
[[ -z "${KEEP_TIME_HOURS}" ]] && KEEP_TIME_HOURS='48'
# 存储空间阈值
STORE_PCT="$4"
[[ -z "${STORE_PCT}" ]] && STORE_PCT='80'
# 是否启用force
[[ ! -z "$5" ]] && FORCE_FLAG="force"
# 操作日志记录
LOG_FILE="${LOG_DIR}/rman_clean_arch_$(date +%Y%m%d%H).log"
# 收集归档目录使用率
ARCH_DEST=$(${ECHO} "set heading off feedback off timing off
select trim(substr(value,10)) from v$parameter where name='log_archive_dest_1';"|sqlplus -S ${SQLPLUS_CONN}|perl -lane 'print if !/@|:|^$/'|tail -1)
# ASM环境：获取ASM 归档目录使用率
ARCH_DEST_PCT=$(${ECHO} "set heading off feedback off
select trim(round((total_mb-usable_file_mb)/total_mb*100,0)) pct from v$asm_diskgroup_stat where name = (select nvl(substr(substr(value,1,instr(value,'/',1,1)-1),11),substr(value,11)) from v$parameter where name ='log_archive_dest_1');"|sqlplus -S ${SQLPLUS_CONN}|perl -lane 'print if !/@|:|^$/'|tail -1)
# 为空则为普通文件系统的占用百分比
if [[ -z "${ARCH_DEST_PCT}" ]]; then
# 处理归档目录非单独文件系统情况
while true;do
df -P |grep -iq "${ARCH_DEST}$"
[[ "$?" = "0" ]] && break || ARCH_DEST=$(dirname ${ARCH_DEST})
done
ARCH_DEST_PCT=$(df -P |grep -i "${ARCH_DEST}$" |awk '{print $5*1}')
fi
if [[ "${ARCH_DEST_PCT}" -ge "${STORE_PCT}" ]]; then
${ORACLE_HOME}/bin/rman log=${LOG_FILE} append <<-EOF
export NLS_DATE_FORMAT='yyyy-mm-dd HH24:MI:SS';
connect target ${RMAN_CONN}
run{
sql "alter session set optimizer_mode=RULE";
crosscheck archivelog all;
delete noprompt expired archivelog all;
delete noprompt ${FORCE_FLAG} archivelog until time 'sysdate-${KEEP_TIME_HOURS}/24';
}
EOF
fi
# select name,sequence#,status ,first_time from v$archived_log order by sequence# desc;
# STATUS=A Available，表示归档日志文件有效
# STATUS=U Unavailable，表示归档日志文件无效
# STATUS=D Deleted，表示归档日志已经被删除
# STATUS=X eXpired，表示归档日志被用户使用操作系统的命令删除了
5.6 开启块跟踪（可选）
su - oracle
sqlplus / as sysdba
SQL> alter database enable block change tracking using file '+DATA';
Database altered.
-- 检查确认
SQL> select status, filename from v$block_change_tracking;
STATUS
----------
FILENAME
--------------------------------------------------------------------------------
ENABLED
+DATA/drmdb/changetracking/ctf.258.990434395
5.7 创建PDB（可选）
-- 创建PDBYYY
sqlplus / as sysdba
alter session set container=CDB$ROOT;
-- alter session set container=PDBORCL;
create pluggable database PDBORCL admin user pdbadmin identified by oracle create_file_dest='+DATA';
-- 2. 打开PDB
alter pluggable database PDBORCL open instances=all;
alter pluggable database PDBORCL save state;
-- 3. 检查
set lines 168 pages 99
col NAME for a12
select inst_id,con_id,name,open_mode from gv$pdbs where name='PDBORCL';
set lines 168 pages 168
col con_name for a18
col instance_name for a18
col restricted for a12
select con_id,con_name, instance_name,state,restricted from dba_pdb_saved_states where con_name='PDBORCL' order by 1;
-- 4. 修改数据库字符集
alter pluggable database PDBYYY close immediate instances=all;
alter pluggable database PDBYYY open read write restricted;
alter session set container=PDBORCL; -- 切换对应的PDB
select userenv('language') from dual;
show con_id con_name
alter database character set internal_use zhs16gbk;
alter pluggable database PDBORCL close;
alter pluggable database PDBORCL open instances=all;
-- alter pluggable database PDBYYY save state;
-- 5. 检查修改结果
alter session set container=PDBORCL;
set lines 168 pages 168
col parameter for a30
col value for a30
select * from nls_database_parameters;
-- 6. 删除PDB
DROP PLUGGABLE DATABASE PDBORCL
INCLUDING DATAFILES;
5.8 资源管理计划(可选)
-- 设置PDB资源限制
su - oracle
alter session set container=PDBYYY;
-- ALTER SYSTEM SET db_performance_profile='MEDIUM' SCOPE=SPFILE PDB='PDBYYY';
alter system set db_performance_profile=medium scope=spfile;
alter pluggable database PDBYYY close immediate instances=all;
alter pluggable database PDBYYY open instances=all;
-- 检查
col name for a28
select inst_id, name, con_id, value, ispdb_modifiable
from gv$system_parameter2 where name = 'db_performance_profile';
5.9 创建表空间
-- 连接到PDB
sqlplus / as sysdba
alter session set container=PDBYYY;
or
sqlplus pdbadmin/wQcEe6gUKo0qz1T@192.168.10.168/pdbyyy
-- 创建表空间
create tablespace main datafile '+DATA' size 32764M autoextend off;
alter tablespace main add datafile '+DATA' size 32764M autoextend off;
-- 检查
col file_name for a82
select FILE_ID,FILE_NAME,TABLESPACE_NAME,BYTES/1024/1024 Mb_size,AUTOEXTENSIBLE,MAXBYTES/1024/1024 max_size from dba_data_files;
5.10 创建用户及授权
-- 连接到对应的PDB
sqlplus / as sysdba
alter session set container=PDBYYY;
or
sqlplus pdbadmin/wQcEe6gUKo0qz1T@192.168.10.168/pdbyyy
-- 创建用户
create user YYY identified by "yy" default tablespace main quota unlimited on main;
-- 授权
grant connect to YYY;
grant resource to YYY;
grant create view to YYY;
-- 检查
col username for a18
col ACCOUNT_STATUS for a18
col PROFILE for a12
select USERNAME,ACCOUNT_STATUS,DEFAULT_TABLESPACE,TEMPORARY_TABLESPACE,LOCAL_TEMP_TABLESPACE,PROFILE from dba_users where username='YYY';
5.11 配置PDB保存状态（可选）
默认情况，pdb需要在cdb启动后，命令方式启动PDB。当配置保存PDB状态后，每当CDB重启后自动将PDB拉起来。
1) 命令配置启动pdb随cdb启动
-- 12.1.0.1版本
# 使用SYS用户创建如下触发器自动启动PDB Pluggable Database：
conn / as sysdba
CREATE TRIGGER open_all_pdbs
AFTER STARTUP
ON DATABASE
BEGIN
EXECUTE IMMEDIATE 'alter pluggable database all open';
END open_all_pdbs;
/
-- 12.1.0.2版本提供命令配置启动pdb随cdb启动
alter pluggable database [pdbname] save state;
alter pluggable database ORCLPDB save state instances=all;
or
alter pluggable database all save state instances=all;
-- 结果检查
set lines 168 pages 999
col CON_NAME for a18
col INSTANCE_NAME for a18
col RESTRICTED for a12
select con_id,CON_NAME,INSTANCE_NAME,STATE,RESTRICTED from dba_pdb_saved_states;
CON_ID CON_NAME INSTANCE_NAME STATE RESTRICTED
---------- ------------------ ------------------ -------------- ------------
3 ORCLPDB orcl2 OPEN NO
3 ORCLPDB orcl1 OPEN NO
SQL>
# 禁用pdb随cdb启动
alter pluggable database ORCLPDB discard state;
SQL> alter pluggable database ORCLPDB discard state;
Pluggable database altered.
SQL> select con_id,CON_NAME,INSTANCE_NAME,STATE,RESTRICTED from dba_pdb_saved_states;
CON_ID CON_NAME INSTANCE_NAME STATE RESTRICTED
---------- ------------------ ------------------ -------------- ------------
3 ORCLPDB orcl2 OPEN NO
SQL> alter pluggable database ORCLPDB save state;
Pluggable database altered.
SQL> select con_id,CON_NAME,INSTANCE_NAME,STATE,RESTRICTED from dba_pdb_saved_states;
CON_ID CON_NAME INSTANCE_NAME STATE RESTRICTED
---------- ------------------ ------------------ -------------- ------------
3 ORCLPDB orcl2 OPEN NO
3 ORCLPDB orcl1 OPEN NO
SQL>
5.12 PDB 启动关闭
alter pluggable database [pdbname] [open|close] [instances=('instance_name'|'instance_name1')]
alter pluggable database ALBINPDB close INSTANCES=('o12c');
alter pluggable database ALBINPDB open INSTANCES=('o12c');
5.13 在PDB级创建AWR
-
在PDB级别设置awr_pdb_autoflush_enabled=true
-
alter session set container=PDB1; alter system set awr_pdb_autoflush_enabled=true;
-
-
正确设置AWR快照
-
select * from cdb_hist_wr_control; DBID SNAP_INTERVAL RETENTION TOPNSQL CON_ID 2580889417 +40150 00:01:00.0 +00008 00:00:00.0 DEFAULT 3 execute dbms_workload_repository.modify_snapshot_settings(interval => 60); select * from cdb_hist_wr_control; DBID SNAP_INTERVAL RETENTION TOPNSQL CON_ID 2580889417 +00000 01:00:00.0 +00008 00:00:00.0 DEFAULT 3
-
-
将AWR_SNAPSHOT_TIME_OFFSET设置为1000000，以避免多个PDB同时创建快照时出现性能问题
alter system set AWR_SNAPSHOT_TIME_OFFSET=1000000 scope=both;
-
生成快照
select * from awr_pdb_snapshot;
-
创建AWR报告
@?/rdbms/admin/awrrpt
5.14 对于12.2非CDB环境，建议调整参数_cursor_obsolete_threshold
- 从12.2数据库版本开始参数_cursor_obsolete_threshold的值增加到8192。
- 这会导致父游标不会被废弃，因此父级下的子游标会扩展到1024以上（这是12.1中的默认值），从而导致cursor mutex的并发问题。
参数的默认值
11.2.0.3: _cursor_obsolete_threshold=100
11.2.0.4: _cursor_obsolete_threshold=1024
12.1: _cursor_obsolete_threshold=1024
12.2: _cursor_obsolete_threshold=8192
从12.2开始，_cursor_obsolete_threshold的默认值大幅增加（从1024开始为8192）以便支持4096个PDB（而12.1只有252个PDB）。 此参数值是在多租户环境中废弃父游标的最大限制，并且不能超过8192。
但这个设置并不适用于非CDB环境，因此对于那些数据库，此参数应手动设置为12.1的默认值，即1024. 默认值1024适用于非CDB环境，并且如果出现问题，可以调整相同的参数，应视具体情况而定。
优化建议
对于12.2非CDB环境，请将参数设置为以下值，以避免由于高版本数而导致的mutex并发问题
alter system set "_cursor_obsolete_threshold"=1024 scope=spfile;
6. 常见问题
6.1 SSH
互信问题
1. 检查/etc/hosts 文件中主机名建议都使用小写字母，scan名称不含有数字，且小于15个字符。
2. 检查防火墙和selinux是否都关闭
3. 检查`ssh`,`scp`等命令工具的路径是否合适
4. 检查$HOME/.ssh目录及其下文件权限
5.
OpenSSH版本8.X
现象
当尝试通过执行 <gridSetup.sh> 配置 19c grid infrastructure 的时候，在 SSH connectivity 步骤，发生如下错误：
[INS-06006] Passwordless SSH connectivity not set up between the following node(s): [
以上错误无法被忽略，因此 CRS 安装会失败。
然而，SSH 配置是成功的，并且 ssh
通过 debug 模式执行 gridSetup.sh:
$ gridSetup.sh -debug | tee /tmp/gridsetup.log
在 debug 的 trace 文件 "/tmp/gridsetup.log" 中，会发现在调用
[Worker 0] [ 2019-05-31 14:40:49.921 CST ] [UnixSystem.remoteCopyFile:848] UnixSystem: **/usr/local/bin/scp -p :'/tmp/GridSetupActions2019-05-31_02-39-46PM/CVU_19.0.0.0.0_grid/scratch/getFileInfo12906.out' /tmp/GridSetupActions2019-05-31_02-39-46PM/.getFileInfo12906.out**
[Thread-440] [ 2019-05-31 14:40:49.921 CST ] [StreamReader.run:62] In StreamReader.run
[Worker 0] [ 2019-05-31 14:40:49.921 CST ] [RuntimeExec.runCommand:294] runCommand: Waiting for the process
[Thread-439] [ 2019-05-31 14:40:49.921 CST ] [StreamReader.run:62] In StreamReader.run
[Thread-440] [ 2019-05-31 14:40:50.109 CST ] [StreamReader.run:66] **ERROR>protocol error: filename does not match request**
[Worker 0] [ 2019-05-31 14:40:50.109 CST ] [RuntimeExec.runCommand:296] runCommand: process returns 1
[Worker 0] [ 2019-05-31 14:40:50.109 CST ] [RuntimeExec.runCommand:323] RunTimeExec: error>
[Worker 0] [ 2019-05-31 14:40:50.109 CST ] [RuntimeExec.runCommand:326] **protocol error: filename does not match request**
原因
OpenSSH 被升级到 8.x. 请注意 OpenSSH 在不同的平台 / OS 可能会有不同的行为，例如在 AIX 平台，OpenSSH 7.5 有这个问题，在 SLES Linux 12 SP4 上，OpenSSH_7.2p2 有这个问题。
# ssh -V
OpenSSH_8.0p1, OpenSSL 1.0.2r 26 Feb 2019
如下的命令在 OpenSSH 8.0 上可能会产生与上面相同的错误。
# scp -p <racnode2>:"'/tmp/test.txt'" /tmp/test.txt
protocol error: filename does not match request
以上错误可以通过在命令中添加 "-T" 选项来避免：
# scp -T -p <racnode2>:"'/tmp/test.txt'" /tmp/test.txt
test.txt 100% 2 0.1KB/s 00:00
为了降低 (CVE-2019-6111) 的风险，OpenSSH 8.0 增加了客户端检查，检查从服务器发送的文件名是否与命令行请求匹配，如果客户端和服务器通配符扩展存在差异，那么客户端可能会拒绝来自服务器的文件。由于这个原因，OpenSSH 8.0 为 scp 提供了一个新的 -T 选项，它将禁用这些客户端检查。
处理
变通方案:
在安装之前 通过 root 用户: (注意如果您的 "scp" 命令的路径与如下示例不同，请根据实际情况修改)
# 将原来的 scp 命令改名.
mv /usr/bin/scp /usr/bin/scp.orig
# 创建一个新的文件 </usr/bin/scp>.
vi /usr/bin/scp
# 在新创建的文件 </usr/bin/scp> 中添加如下行.
/usr/bin/scp.orig -T $*
# 修改文件的权限.
chmod 555 /usr/bin/scp
在安装结束后:
mv /usr/bin/scp.orig /usr/bin/scp
参考文档
在 OpenSSH 升级到 8.x 后 GI 安装失败 INS-06006 (Doc ID 2639907.1)
6.2 19C需要将节点1上oui-patch.xml传递到其它节点
为了解决其它节点因缺少oui-patch.xml文件导致补丁升级失败，因此需要将其传递到其它所有节点并确保文件权限。
# 1. 复制oui-patch.xml文件到其它节点，避免后期补丁升级异常
-- “During 19.x GI installation, the file 'oui-patch.xml' will be created under the central inventory directory on the OUI node (node where gridSetup.sh was invoked) but not on the other nodes.
# node1 (执行gridSetup.sh的节点)
export INV_DIR="$(grep 'inventory_loc' /etc/oraInst.loc|awk -F= '{print $NF}')/ContentsXML"
scp ${INV_DIR}/oui-patch.xml o19c2:${INV_DIR}/oui-patch.xml
ssh o19c2 "chmod 660 ${INV_DIR}/oui-patch.xml && chown grid:oinstall ${INV_DIR}/oui-patch.xml"
6.3 低版本客户端连接报错ORA-28040
现象
ORA-28040: No matching authentication protocol
ORA-28040: 没有匹配的验证协议
问题原因
原因客户端与服务端的密码版本（dba_users.password_versions）不一致导致
解决方法
方法1
添加sqlnet.ora文件配置
在数据库服务器上DB软件的$ORACLE_HOME/network/admin/sqlnet.ora文件添加相应参数
# 12c以下版本
# SQLNET.ALLOWED_LOGON_VERSION=8
# 12c及以上版本
SQLNET.ALLOWED_LOGON_VERSION_SERVER=8
SQLNET.ALLOWED_LOGON_VERSION_CLIENT=8
- QLNET.ALLOWED_LOGON_VERSION_SERVER：控制可以连接到12c数据库的客户端版本（client -->12c server ）
- SQLNET.ALLOWED_LOGON_VERSION_CLIENT：控制12c数据库可以连到哪些版本的数据库（12c server -->其它版本dbserver），例如：控制通过DB LINK可连接到哪些版本的oracle库。
注：单实例或RAC都是此目录的sqlnet.ora文件
重新修改用户密码
方法2
参考文档
不同 Oracle 版本的客户端/服务器互操作性支持矩阵 (Doc ID 1945782.1)
12c Database Alert.log File Shows The Message: Using Deprecated SQLNET.ALLOWED_LOGON_VERSION Parameter (Doc ID 2111876.1)
6.4 在19C版本上cssd访问使用了AFD的Voting disk时节点重启
现象
- 2节点的19c的GI 安装的时候，运行root.sh失败，并提示以下错误，而且第一个节点被踢出集群，安装界面消失。 节点可以正常再启动。
- 使用了Oracle ASM Filter Driver特性
原因
Bug 30006262
出现在使用AFD的系统升级19c的过程中
处理
-
补丁30006262的安装文档，在所有的节点安装补丁
-
重新运行root.sh脚本
-
如果是19C以下的环境要升级到19c，并且使用了AFD的功能，请先安装这个补丁到19c的 Oracle Home里，然后再运行：rootupgrade.sh避免该问题的出现。
参考文档
19C: cssd访问使用了AFD的Voting disk时节点重启 (Doc ID 2639902.1)
6.5 asm mmon进程跟踪文件增大问题
现象
MMON跟踪文件增大，并被消息“dbkrfssm：mmon not open”淹没。该消息每隔5分钟在跟踪文件中重复一次。
[grid@node1 ~]$ ps -ef|grep 'asm_mmon' |grep -v grep |awk '{print $2}' |xargs lsof -p |grep ".trc$" |awk '{print $NF}'
/ups/oracle/grid/diag/asm/+asm/+ASM1/trace/+ASM1_mmon_14560.trc
dbkrfssm: mmon not open
*** 2020-11-12T09:12:19.222584+08:00
dbkrfssm: mmon not open
*** 2020-11-12T09:17:19.259359+08:00
dbkrfssm: mmon not open
dbkrfssm: mmon not open
*** 2020-11-12T09:27:19.341368+08:00
dbkrfssm: mmon not open
*** 2020-11-12T09:32:19.420488+08:00
dbkrfssm: mmon not open
dbkrfssm: mmon not open
原因
Bug 28370061 : ASM MMON EVERY 5 MIN WRITES TRC MESSAGE 'DBKRFSSM: MMON NOT OPEN'
处理
- 在GI_HOME应用补丁28370061，以解决此问题
- 临时解决方法，可以手动删除跟踪文件。
参考文档
ASM mmon每5分钟写入一次trc消息'dbkrfssm：mmon not open' (Doc ID 2610739.1)
6.6 diagsnap进程收集stack traces造成进程hang和节点驱逐
Diagsnap是从12.1.0.2 GI开始引入的，CHM的osysmod 管理着diagsnap。 diagsnap用来收集那些CHM不收集的额外的os的statistics。
Diagsnap每15分钟自动执行收集baseline的matric。此外以下的事件也会触发diagsnap的matric收集。
-
cssd报告missing network heartbeats(NHB's).
-
gipcd 侦测到一个或者多个网卡启动或者停止.
-
gipcd rank的事件（网卡的健康状态，GIPC会把启动阶段标记为-1，没有packet传输标记为0，任何小于90的标记意味着有packet的丢失）
问题触发条件
- Oracle Clusterware 版本是 12.1.0.2.160419或更高版本的12.1.0.2, 或12.2.0.1, 或 18
- 启用了diagsnap。 注意：即使曾显式地禁用过diagsnap 的12.1.0.2 和12.2.0.1 环境，其实际运行的是比2017年10月 (171017)早的季度release时，当它升级到 2017年10月的release后，diagsnap会被重新启用
- 从 Oracle Clusterware 12.1.0.2.171017, 12.2.0.1.171017, 和18版开始 ，除diagsnap之外，还有其它组件也会生成stack traces
处理
# 1. 禁用diagsnap
<GRID_HOME>/bin/oclumon manage -disable diagsnap
# 2.禁用pstack
<GRID_HOME>/bin/oclumon manage -disable pstack
或 编辑$GI_HOME/crf/admin/crf
# 1. 关闭资源
# crsctl stop res ora.crf -init
# 2. vi $GI_HOME/crf/admin/crf$(hostname -s).ora
DIAGSNAP=DISABLE
PSTACK=DISABLE
# 3. 启动资源
# crsctl start res ora.crf -init
参考文档
diagsnap 和其它组件对一些clusterware进程收集stack traces造成进程hang和节点驱逐 (Doc ID 2440134.1)
Document 2469642.1 ALERT: 在12.1.0.2和12.2的GI Home安装完2017年10月到2018年7月之间的PSU/RU需要禁用diagsnap中的pstack
参考 <Document 27068526.8> 和 <Document 2251437.1> 里的更多详情
6.7 运行完 root 脚本之后，如果 installer GUI 窗口失效处理
在运行完 root 脚本（root.sh or rootupgrade.sh）之后，如果 installer GUI 窗口失效，可以执行如下命令来完成 plugin：
gridSetup.sh -executeConfigTools
6.8 OL7|RHEL7中systemd自启动或关闭服务
# vi /usr/lib/systemd/system/dbora.service
[Unit]
Description=Oracle Database Start/Stop Service
After=syslog.target network.target local-fs.target remote-fs.target
[Service]
# systemd, by design does not honor PAM limits
# See: https://bugzilla.redhat.com/show_bug.cgi?id=754285
LimitNOFILE=65536
LimitNPROC=16384
LimitSTACK=32M
LimitMEMLOCK=infinity
LimitCORE=infinity
Type=simple
User=oracle
Group=oinstall
Restart=no
ExecStartPre=/bin/rm -rf /u01/app/oracle/product/12.2.0/dbhome_1/listener.log
ExecStartPre=/bin/rm -rf /u01/app/oracle/product/12.2.0/dbhome_1/startup.log
ExecStart=/u01/app/oracle/product/12.2.0/dbhome_1/bin/dbstart /u01/app/oracle/product/12.2.0/dbhome_1
RemainAfterExit=yes
ExecStop=/bin/rm -rf /u01/app/oracle/product/12.2.0/dbhome_1/shutdown.log
ExecStop=/u01/app/oracle/product/12.2.0/dbhome_1/bin/dbshut /u01/app/oracle/product/12.2.0/dbhome_1
TimeoutStopSec=5min
[Install]
WantedBy=multi-user.target
6.9 DB alert日志抛出ORA-27300，ORA-27301和ORA-27302
现象
在DB 实例的alert日志中抛出以下错误信息：
2020-11-18T13:42:58.448277+08:00
Errors in file /ups/oracle/database/diag/rdbms/orcl/orcl1/trace/orcl1_asmb_5574.trc (incident=317146) (PDBNAME=CDB$ROOT):
ORA-00603: ORACLE server session terminated by fatal error
ORA-27504: IPC error creating OSD context
ORA-27300: OS system dependent operation:sendmsg failed with status: 105
ORA-27301: OS failure message: No buffer space available
ORA-27302: failure occurred at: sskgxpsnd2
Incident details in: /ups/oracle/database/diag/rdbms/orcl/orcl1/incident/incdir_317146/orcl1_asmb_5574_i317146.trc
2020-11-18T13:43:10.752227+08:00
原因
发生这种情况是因为可用于网络缓冲区保留的空间较少
处理
- On servers with High Physical Memory, the parameter vm.min_free_kbytes should be set in the order of 0.4% of total Physical Memory. This helps in keeping a larger range of defragmented memory pages available for network buffers reducing the probability of a low-buffer-space conditions.
**For example, on a server which is having 256GB RAM, the parameter vm.min_free_kbytes should be set to 1073742 **
/sbin/sysctl -a | grep min_free_kbytes
On NUMA Enabled Systems, the value of vm.min_free_kbytes should be multiplied by the number of NUMA nodes since the value is to be split across all the nodes.
On NUMA Enabled Systems, the value of vm.min_free_kbytes = n * 0.4% of total Physical Memory. Here 'n' is the number of NUMA nodes.
- Additionally, the MTU value should be modified as below
Linux : #ifconfig lo mtu 16384
AIX : #chdev -Pl lo0 -a mtu=16436
To make the change persistent over reboot add the following line in the file /etc/sysconfig/network-scripts/ifcfg-lo :
MTU=16436
Save the file and restart the network service to load the changes
#service network restart
检查确认
netstat -in
Note : While making the changes in CRS nodes, if network is restarted while CRS is up, it can hung CRS. So cluster services should be stopped prior to the network restart.
参考文档
ODA Upgrade to 12.1.2.11 or .12 Causes ORA-27301 ORA-27302: failure occurred at: sskgxpsnd2 "The OS has most likely run out of buffers (rval: 4)" with Multiple asmcmd daemon (Doc ID 2484025.1)
"ORA-27301: OS failure message: No buffer space available" occurs on OPC RAC (Doc ID 2397062.1)
Oracle Linux: ORA-27301:OS Failure Message: No Buffer Space Available (Doc ID 2041723.1)
附录
https://www.cnblogs.com/plluoye/p/10963194.html
http://blog.itpub.net/31439444/viewspace-2679289/
How to Complete Grid Infrastructure Configuration Assistant(Plug-in) if OUI is not Available (Doc ID 1360798.1)
https://www.cndba.cn/dave/article/310
hugepages配置
开启大页
- Calculate the amount of hugepages that will need to be allocated for your Oracle instance.
NUMBER_OF_HUGEPAGES = ( SGA + PGA + (20KB * # of Oracle_processes)) / 2MB
-
SGA and PGA are values that are obtained from the Oracle instance. The correct number here is crucial. If you do not allocate enough hugepages for use with Oracle, zero pages will be used, causing "lost" memory from your available pool.
-
Enable the oracle user to be able to utilize huge pages in the /etc/sysctl.conf file.
vm.hugetlb_shm_group=`id -g oracle`
- Set the amount of huge pages to allocate in the /etc/sysctl.conf file.
vm.nr_hugepages=NUMBER_OF_HUGEPAGES
Note: NUMBER_OF_HUGEPAGES should be replaced with the number calculated in step 1.
- Calculate the amount of pages to put into the 'memlock' parameter.
AMOUNT_OF_MEMORY = NUMBER_OF_HUGEPAGES * 1024 * 2
- Set the 'memlock' parameter in the /etc/security/limits.conf file.
oracle - memlock AMOUNT_OF_MEMORY
Note: AMOUNT_OF_MEMORY should be replaced with the value calculated from step 5.
- Restart the system.
- These settings can be changed on the fly in a live environment; however, memory defragmentation will begin to occur which will have a negative impact on the system and could result in the system being unresponsive for an extended period of time while a large block of free memory is allocated. It is recommended to restart the machine completely to avoid this from happening.
- If you're having trouble calculating your correct amount of huge pages, you may be able to use the following script to automatically determine the correct value
#!/bin/bash
KERN=`uname -r | awk -F. '{ printf("%d.%d
",$1,$2); }'`
# Find out the HugePage size
HPG_SZ=`grep Hugepagesize /proc/meminfo | awk '{print $2}'`
# Start from 1 pages to be on the safe side and guarantee 1 free HugePage
NUM_PG=1
# Cumulative number of pages required to handle the running shared memory segments
for SEG_BYTES in `ipcs -m | awk '{print $5}' | grep "[0-9][0-9]*"`
do
MIN_PG=`echo "$SEG_BYTES/($HPG_SZ*1024)" | bc -q`
if [ $MIN_PG -gt 0 ]; then
NUM_PG=`echo "$NUM_PG+$MIN_PG+1" | bc -q`
fi
done
# Finish with results
case $KERN in
'2.4') HUGETLB_POOL=`echo "$NUM_PG*$HPG_SZ/1024" | bc -q`;
echo "Recommended setting: vm.hugetlb_pool = $HUGETLB_POOL" ;;
'2.6') echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;;
'3.10')echo "Recommended setting: vm.nr_hugepages = $NUM_PG";;
*) echo "Unrecognized kernel version $KERN. Exiting." ;;
esac
检查大页的使用情况
-
First, check for the current hugepages usage:
[root@server ~]# grep HugePages_ /proc/meminfo HugePages_Total: 16299 HugePages_Free: 7764 HugePages_Rsvd: 5330 HugePages_Surp: 0 -
Now, let's deduct the free pages, so we can find the used pages and sum to it the reserved pages. So the allocated pages is going to be
Total - Free + Rsvd16299 - 7764 + 5330 = 13865 -
Multiply the alloocated pages by 2048 (2048 is the number of kbytes of a single page)
13865 x 2048 = 28,395,520 kbytes -
If you want to put the kbytes in byte count, multiply it by 1024 (1 kbyte = 1024 bytes):
28395520 x 1024 = 29,077,012,480 bytes
Where is the memory
- You can easily quantify the shared hugepages memory. To do so:
-
List the shared memory in use:
[root@rfreire sys]# ipcs -m ------ Shared Memory Segments -------- key shmid owner perms bytes nattch status 0x00000000 163840 oracle 640 14680064 50 0x00000000 196609 oracle 640 2499805184 50 0x27126a4c 229378 oracle 640 2097152 50 0x00000000 5636099 oracle 640 33554432 58 0x00000000 5668868 oracle 640 4160749568 58 [...] -
Sum the 5th column (quick shell:
ipcs -m|awk '{ print $5}'|awk '{a+=$0}END{print a}') and match against/proc/meminfohugepages information (see procedure above). If the values matches, then you have a hugepages-only shared memory. If you get a larger value, you have 4kb regular pages shared memory in use as well. Just deduct this sum from/proc/meminfohugepages value in bytes and then you'll find how much you have of regular 4kb pages.
-
Check if you are using a hugepages filesystem. Grep for huge in
/proc/mounts:[root@server ~]# grep -i huge /proc/mounts none /hugepages hugetlbfs rw,relatime 0 0 -
Unfortunately, at this time there are no means to quantify private hugetlbfs pages, which are used for
qemu-kvm, for example.
How to caculate the size of hugepage used by a specified process
- The following command can be used to caculate the size of hugepage used by a specified process, assumption that HugePage size is 2048 kB, the output unit is MiB:
grep -B 11 'KernelPageSize: 2048 kB' /proc/[PID]/smaps | grep "^Size:" | awk 'BEGIN{sum=0}{sum+=$2}END{print sum/1024}'
- Note: avoid double counting of the same address in /proc/[PID]/smaps.