客户一套RAC环境是华为的存储,共享盘是/dev/sd*,咋一看还怀疑是没有进行多路径配置,实际和主机工程师是已经配置好的,我们使用upadmin show vlun
命令可以查看到:
[root@xxdb01 ~]# upadmin show vlun
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Vlun ID Disk Name Lun WWN Status Capacity Ctrl(Own/Work) Array Name Dev Lun ID No. of Paths(Available/Total)
0 sdb LUN_Oracle_400G_0000 6acb3b510041191b0de7bcdd0000000f Normal 400.00GB 0D/0D Huawei.18500V5 15 8/8
1 sdc LUN_Oracle_400G_0001 6acb3b510041191b0de7be3900000010 Normal 400.00GB 0A/0A Huawei.18500V5 16 8/8
2 sdd LUN_Oracle_400G_0002 6acb3b510041191b0de7bec100000011 Normal 400.00GB 0B/0B Huawei.18500V5 17 8/8
3 sde LUN_Oracle_400G_0003 6acb3b510041191b0de7bfc900000012 Normal 400.00GB 0C/0C Huawei.18500V5 18 8/8
4 sdf LUN_Oracle_400G_0004 6acb3b510041191b0de7c03f00000013 Normal 400.00GB 0D/0D Huawei.18500V5 19 8/8
5 sdg LUN_Oracle_400G_0005 6acb3b510041191b0de7c09e00000014 Normal 400.00GB 0A/0A Huawei.18500V5 20 8/8
6 sdh LUN_Oracle_400G_0006 6acb3b510041191b0de7c0e900000015 Normal 400.00GB 0B/0B Huawei.18500V5 21 8/8
7 sdi LUN_Oracle_400G_0007 6acb3b510041191b0de7c12e00000016 Normal 400.00GB 0C/0C Huawei.18500V5 22 8/8
8 sdj LUN_Oracle_5G_0003 6acb3b510041191b0de893a0000000da Normal 5.00GB 0C/0C Huawei.18500V5 218 8/8
9 sdk LUN_Oracle_5G_0004 6acb3b510041191b0de8941e000000db Normal 5.00GB 0D/0D Huawei.18500V5 219 8/8
10 sdl LUN_Oracle_5G_0005 6acb3b510041191b0de894a4000000dc Normal 5.00GB 0A/0A Huawei.18500V5 220 8/8
11 sdm LUN_Oracle_100G_0002 6acb3b510041191b0de80f1f00000039 Normal 100.00GB 0B/0B Huawei.18500V5 57 8/8
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[root@xxdb01 ~]#
实际上直接使用这些盘也是可以的,但是考虑到规范,参考之前客户udev绑定规则规范:
--not available
KERNEL=="sd*",BUS=="scsi",PROGRAM=="/sbin/scsi_id i --whitelisted --device=/dev/$name",RESULT=="36000c29b263ed2452f80e9848bdf2fa5",NAME="asm-2g-2fa5-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
可以看到别名的命名方式是:asm-lunsize-id后四位-磁盘组名+编号。这样以后遇到加盘/删盘这类操作,就能快速帮助DBA确认。
不过因为上面这个udev语法是RHEL 6的,在CentOS 7并不适用,换成7对应的语法就是:
--ok!
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7bcdd0000000f",SYMLINK+="asm-400g-000f-data1",OWNER="grid",GROUP="asmadmin",MODE="0660"
考虑到盘相对多,一个个写既费时又容易出错,想起早些年间装RAC时,经常会参考maclean的一个方法,就是把这个工作写一个脚本:
vi /u01/asmdisk.sh
for i in b c d e f g h i j k l m;
do
echo "KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="`/lib/udev/scsi_id -g -u -d /dev/sd$i`",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660""
done
执行脚本:sh /u01/asmdisk.sh ,结果为:
--script-result
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7bcdd0000000f",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7be3900000010",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7bec100000011",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7bfc900000012",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7c03f00000013",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7c09e00000014",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7c0e900000015",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7c12e00000016",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de893a0000000da",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de8941e000000db",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de894a4000000dc",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de80f1f00000039",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
使用带列编辑的文本编辑器快速修改如下,然后复制到/etc/udev/rules.d/99-oracle-asmdevices.rules
配置文件中:
--modify
[root@xxdb01 ~]# cat /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7bcdd0000000f",SYMLINK+="asm-400g-000f-data1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7be3900000010",SYMLINK+="asm-400g-0010-data2",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7bec100000011",SYMLINK+="asm-400g-0011-data3",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7bfc900000012",SYMLINK+="asm-400g-0012-data4",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7c03f00000013",SYMLINK+="asm-400g-0013-data5",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7c09e00000014",SYMLINK+="asm-400g-0014-data6",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7c0e900000015",SYMLINK+="asm-400g-0015-data7",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7c12e00000016",SYMLINK+="asm-400g-0016-data8",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de893a0000000da",SYMLINK+="asm-5g-00da-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de8941e000000db",SYMLINK+="asm-5g-00db-grid2",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de894a4000000dc",SYMLINK+="asm-5g-00dc-grid3",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de80f1f00000039",SYMLINK+="asm-100g-0039-arch1",OWNER="grid",GROUP="asmadmin",MODE="0660"
这里看到这个/lib/udev/scsi_id -g -u -d /dev/sd*
查到的结果和存储多路径查到的Lun WWN,除了scsi_id查询的结果首位多一个3,后面是完全一样的。
此时可以使用udevadm进行应用规则:
udevadm control --reload
udevadm trigger
然后查看结果:
[root@xxdb01 ~]# ls -l /dev/asm*
lrwxrwxrwx. 1 root root 3 Sep 8 10:19 /dev/asm-100g-0039-arch1 -> sdm
lrwxrwxrwx. 1 root root 3 Sep 8 10:19 /dev/asm-400g-000f-data1 -> sdb
lrwxrwxrwx. 1 root root 3 Sep 8 10:19 /dev/asm-400g-0010-data2 -> sdc
lrwxrwxrwx. 1 root root 3 Sep 8 10:19 /dev/asm-400g-0011-data3 -> sdd
lrwxrwxrwx. 1 root root 3 Sep 8 10:19 /dev/asm-400g-0012-data4 -> sde
lrwxrwxrwx. 1 root root 3 Sep 8 10:19 /dev/asm-400g-0013-data5 -> sdf
lrwxrwxrwx. 1 root root 3 Sep 8 10:19 /dev/asm-400g-0014-data6 -> sdg
lrwxrwxrwx. 1 root root 3 Sep 8 10:19 /dev/asm-400g-0015-data7 -> sdh
lrwxrwxrwx. 1 root root 3 Sep 8 10:19 /dev/asm-400g-0016-data8 -> sdi
lrwxrwxrwx. 1 root root 3 Sep 8 10:19 /dev/asm-5g-00da-grid1 -> sdj
lrwxrwxrwx. 1 root root 3 Sep 8 10:19 /dev/asm-5g-00db-grid2 -> sdk
lrwxrwxrwx. 1 root root 3 Sep 8 10:19 /dev/asm-5g-00dc-grid3 -> sdl
[root@xxdb01 ~]# ls -l /dev/sd*
brw-rw----. 1 root disk 8, 0 Sep 8 10:10 /dev/sda
brw-rw----. 1 root disk 8, 1 Sep 8 10:10 /dev/sda1
brw-rw----. 1 root disk 8, 2 Sep 8 10:10 /dev/sda2
brw-rw----. 1 grid asmadmin 8, 16 Sep 8 16:02 /dev/sdb
brw-rw----. 1 grid asmadmin 8, 32 Sep 8 10:19 /dev/sdc
brw-rw----. 1 grid asmadmin 8, 48 Sep 8 10:19 /dev/sdd
brw-rw----. 1 grid asmadmin 8, 64 Sep 8 10:19 /dev/sde
brw-rw----. 1 grid asmadmin 8, 80 Sep 8 10:19 /dev/sdf
brw-rw----. 1 grid asmadmin 8, 96 Sep 8 10:19 /dev/sdg
brw-rw----. 1 grid asmadmin 8, 112 Sep 8 10:19 /dev/sdh
brw-rw----. 1 grid asmadmin 8, 128 Sep 8 10:19 /dev/sdi
brw-rw----. 1 grid asmadmin 8, 144 Sep 8 16:02 /dev/sdj
brw-rw----. 1 grid asmadmin 8, 160 Sep 8 16:02 /dev/sdk
brw-rw----. 1 grid asmadmin 8, 176 Sep 8 16:02 /dev/sdl
brw-rw----. 1 grid asmadmin 8, 192 Sep 8 16:02 /dev/sdm
最终使用asmca创建磁盘组,最终结果为:
[grid@xxdb01 ~]$ asmcmd lsdg
State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N 512 4096 4194304 102400 102276 0 102276 0 N ARCH/
MOUNTED EXTERN N 512 4096 4194304 3276800 3276620 0 3276620 0 N DATA/
MOUNTED NORMAL N 512 4096 4194304 15360 14320 5120 4600 0 Y GRID/
[grid@xxdb01 ~]$
可以看到,磁盘组的名字和上面定义的磁盘别名,可以很好的进行管理维护,个人认为这个客户的规范很值得借鉴。