可以通过2种方式查看HBA的WWN信息:
1. 查看sys文件系统
查看HBA卡型号:
[root@localhost ~]# lspci | grep -i fibre
13:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
13:00.1 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
查看HBA卡WWN信息:
[root@localhost ~]# ls -l /sys/class/fc_host/ total 0 drwxr-xr-x 3 root root 0 Jun 7 23:55 host3 drwxr-xr-x 3 root root 0 Jun 7 23:58 host4 [root@localhost ~]# cat /sys/class/fc_host/host3/port_name 0x21000024ff462814
2. 采用systool
systool所在软件包:
yum -y install sysfsutils
systool -c fc_host -v -c <class_name> / -b <bus_name> / -m <module_name> [root@localhost ~]# systool -v -c fc_host Class = "fc_host" Class Device = "host3" Class Device path = "/sys/class/fc_host/host3" fabric_name = "0x1000000533746b8f" issue_lip = <store method only> node_name = "0x20000024ff462814" port_id = "0x011300" port_name = "0x21000024ff462814" port_state = "Online" port_type = "NPort (fabric via point-to-point)" speed = "8 Gbit" supported_classes = "Class 3" supported_speeds = "1 Gbit, 2 Gbit, 4 Gbit, 8 Gbit" symbolic_name = "QLE2562 FW:v5.03.16 DVR:v8.03.07.03.05.07-k" system_hostname = "" tgtid_bind_type = "wwpn (World Wide Port Name)" uevent = <store method only> Device = "host3" Device path = "/sys/devices/pci0000:00/0000:00:07.0/0000:13:00.0/host3" ct = edc = <store method only> els = fw_dump = nvram = "ISP " optrom_ctl = <store method only> optrom = reset = <store method only> sfp = "" uevent = <store method only> vpd = "‚(" Class Device = "host4" Class Device path = "/sys/class/fc_host/host4" fabric_name = "0x10000005337638b9" issue_lip = <store method only> node_name = "0x20000024ff462815" port_id = "0x021300" port_name = "0x21000024ff462815" port_state = "Online" port_type = "NPort (fabric via point-to-point)" speed = "8 Gbit" supported_classes = "Class 3" supported_speeds = "1 Gbit, 2 Gbit, 4 Gbit, 8 Gbit" symbolic_name = "QLE2562 FW:v5.03.16 DVR:v8.03.07.03.05.07-k" system_hostname = "" tgtid_bind_type = "wwpn (World Wide Port Name)" uevent = <store method only> Device = "host4" Device path = "/sys/devices/pci0000:00/0000:00:07.0/0000:13:00.1/host4" ct = edc = <store method only> els = fw_dump = nvram = "ISP " optrom_ctl = <store method only> optrom = reset = <store method only> sfp = "" uevent = <store method only> vpd = "‚("
Multipath相关软件介绍
1. device-mapper-multipath
即multipath-tools,主要提供multipathd和multipath等工具和 multipath.conf等配置文件。这些工具通过device mapper的ioctr的接口创建和配置multipath,设备创建的多路径设备映射会在/dev /mapper中。
2. device-mapper
主要包括两大部分:内核部分和用户部分。内核部分主要由device mapper核心(dm.ko)和一些target driver(md-multipath.ko)。核心完成设备的映射,而target根据映射关系和自身特点具体处理从mappered device 下来的i/o。同时,在核心部分,提供了一个接口,用户通过ioctr可和内核部分通信,以指导内核驱动的行为,比如如何创建mappered device,这些divece的属性等。linux device mapper的用户空间部分主要包括device-mapper这个包。其中包括dmsetup工具和一些帮助创建和配置mappered device的库。这些库主要抽象,封装了与ioctr通信的接口,以便方便创建和配置mappered device。multipath-tool的程序中就需要调用这些库
3. dm-multipath.ko和dm.ko:dm.ko是device mapper驱动。它是实现multipath的基础。dm-multipath其实是dm的一个target驱动。
https://www.cnblogs.com/jianqiang2010/archive/2011/09/01/2162574.html
AIX
http://blog.163.com/herod_xiao/blog/static/871883992011819112227689/
1. 检查multipath模块,如果没有相关模块就说明没有安装相关软件包
lsmod | grep dm_multipath
yum -y install device-mapper device-mapper-multipath
2. 加载相关模块
modprobe dm-multipath
modprobe dm-round-robin
3. 生成multipath配置文件,并配置
/sbin/mpathconf --enable
配置multipath.conf文件
4. 启动multipath服务
service multipathd start
chkcofig multipathd on
[root@tyhr ~]# multipath -ll mpathc (36005076801820709e000000000000087) dm-3 IBM,2145 size=300G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active |- 4:0:0:1 sdc 8:32 active ready running |- 4:0:2:1 sde 8:64 active ready running |- 5:0:2:1 sdg 8:96 active ready running `- 5:0:4:1 sdi 8:128 active ready running mpathb (36005076801820709e000000000000086) dm-2 IBM,2145 size=100G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active |- 4:0:0:0 sdb 8:16 active ready running |- 4:0:2:0 sdd 8:48 active ready running |- 5:0:2:0 sdf 8:80 active ready running `- 5:0:4:0 sdh 8:112 active ready running
如果完成配置后,不能使用multipath -ll查看到任何多路径信息,就需要手工重新扫描HBA信息
一般新增LUN映射给主机后,系统无法直接更新挂载的存储盘,需要重新扫描。
# cd /sys/class/scsi_host/host4/ # echo '- - - ' > scan 或 echo "1" > /sys/class/fc_host/host/issue_lip //某些存储或系统没有scan文件,可以通issue_lip文件识别
RHEL5.x 操作系统,新增multipath后,能够识别磁盘,但是通过fdisk分区后,不能显示分区信息(/dev/mapper/dm-app 磁盘不显示/dev/mapper/dm-appp1)。partprob无报错但是没有效果。fdisk w保存后有报错:
WARNING: Re-reading the partition table failed with error 22: Invalid argument.
Run "kpartx -a" after FDISK is completed to add all partition mappings on the newly-created multipath device [root@TYEDI1 host4]# ls -l /dev/mapper/dm-* brw-rw---- 1 root disk 253, 0 Jan 21 22:59 /dev/mapper/dm-app brw-rw---- 1 root disk 253, 2 Jan 21 22:35 /dev/mapper/dm-arch brw-rw---- 1 root disk 253, 1 Jan 21 22:52 /dev/mapper/dm-db [root@TYEDI1 host4]# kpartx -a /dev/mapper/dm-app [root@TYEDI1 host4]# ls -l /dev/mapper/dm-* brw-rw---- 1 root disk 253, 0 Jan 21 22:59 /dev/mapper/dm-app brw-rw---- 1 root disk 253, 3 Jan 21 23:01 /dev/mapper/dm-app1 brw-rw---- 1 root disk 253, 2 Jan 21 22:35 /dev/mapper/dm-arch brw-rw---- 1 root disk 253, 1 Jan 21 22:52 /dev/mapper/dm-db
其他配置:
1. udev绑定权限
RHEL6.5,对于multipath的权限,手工去修改几秒后会变回root。所以需要使用udev去绑定好权限。
1)搜索对应的配置文件模板: [root@jyrac1 ~]# find / -name 12-* /usr/share/doc/device-mapper-1.02.79/12-dm-permissions.rules
2)根据模板新增12-dm-permissions.rules文件在/etc/udev/rules.d/下面: vi /etc/udev/rules.d/12-dm-permissions.rules # MULTIPATH DEVICES # # Set permissions for all multipath devices ENV{DM_UUID}=="mpath-?*", OWNER:="grid", GROUP:="asmadmin", MODE:="660" # Set permissions for first two partitions created on a multipath device (and detected by kpartx) # ENV{DM_UUID}=="part[1-2]-mpath-?*", OWNER:="root", GROUP:="root", MODE:="660" 3)查看多路径对应的底层dm设备: [root@jyrac2 rules.d]# ls -lh /dev/dm* brw-rw---- 1 root disk 253, 0 Jul 19 16:40 /dev/dm-0 brw-rw---- 1 root disk 253, 1 Jul 19 16:40 /dev/dm-1 brw-rw---- 1 root disk 253, 2 Jul 19 21:20 /dev/dm-2 brw-rw---- 1 root disk 253, 3 Jul 19 21:20 /dev/dm-3 4)启动start_udev [root@jyrac2 rules.d]# start_udev Starting udev: [ OK ] 5)查看权限: [root@jyrac2 rules.d]# ls -lh /dev/dm* brw-rw---- 1 root disk 253, 0 Jul 19 21:35 /dev/dm-0 brw-rw---- 1 root disk 253, 1 Jul 19 21:35 /dev/dm-1 brw-rw---- 1 grid asmadmin 253, 2 Jul 19 21:35 /dev/dm-2 brw-rw---- 1 grid asmadmin 253, 3 Jul 19 21:35 /dev/dm-3 6)等30s后再查,权限固定: [root@jyrac2 rules.d]# ls -lh /dev/dm* brw-rw---- 1 root disk 253, 0 Jul 19 21:35 /dev/dm-0 brw-rw---- 1 root disk 253, 1 Jul 19 21:35 /dev/dm-1 brw-rw---- 1 grid asmadmin 253, 2 Jul 19 21:35 /dev/dm-2 brw-rw---- 1 grid asmadmin 253, 3 Jul 19 21:35 /dev/dm-3
可以使用lsscsi查看识别的磁盘信息:
[root@tyhr ~]# lsscsi [0:0:0:0] cd/dvd TSSTcorp DVD-ROM TS-L333H ID03 /dev/sr0 [2:0:0:0] cd/dvd IMM Virtual CD/DVD 0316 /dev/sr1 [3:0:0:0] disk IBM-ESXS MBF2300RC SB27 - [3:0:1:0] disk IBM-ESXS MBF2300RC SB27 - [3:1:1:0] disk LSILOGIC Logical Volume 3000 /dev/sda [4:0:0:0] disk IBM 2145 0000 /dev/sdb [4:0:0:1] disk IBM 2145 0000 /dev/sdc [4:0:2:0] disk IBM 2145 0000 /dev/sdd [4:0:2:1] disk IBM 2145 0000 /dev/sde [5:0:2:0] disk IBM 2145 0000 /dev/sdf [5:0:2:1] disk IBM 2145 0000 /dev/sdg [5:0:4:0] disk IBM 2145 0000 /dev/sdh [5:0:4:1] disk IBM 2145 0000 /dev/sdi
1. multipath常用操作:
multipath -r (修改multipath.conf配置文件之后重新加载)
multipath -ll (查看多路径状态)
multipath -v2 (格式化路径,检测路径,合并路径)
multipath -v3 (查看多路径详情blacklist、whitelist和设备wwid)
multipath -F (删除现有链路)
2. multipath文件详细介绍
接下来的工作就是要编辑/etc/multipath.conf的配置文件
相关参数的标准文档的介绍:
Attribute |
Description |
|||||||||
wwid |
Specifies the WWID of the multipath device to which the multipath attributes apply. This parameter is mandatory for this section of themultipath.conf file. |
|||||||||
alias |
Specifies the symbolic name for the multipath device to which themultipath attributes apply. If you are using user_friendly_names, do not set this value tompathn; this may conflict with an automatically assigned user friendly name and give you incorrect device node names. |
|||||||||
path_grouping_policy |
|
|||||||||
path_selector |
|
|||||||||
failback |
|
|||||||||
prio |
|
|||||||||
no_path_retry |
|
|||||||||
rr_min_io |
Specifies the number of I/O requests to route to a path before switching to the next path in the current path group. This setting is only for systems running kernels older that 2.6.31. Newer systems should userr_min_io_rq. The default value is 1000. |
|||||||||
rr_min_io_rq |
Specifies the number of I/O requests to route to a path before switching to the next path in the current path group, using request-based device-mapper-multipath. This setting should be used on systems running current kernels. On systems running kernels older than 2.6.31, use rr_min_io. The default value is 1. |
|||||||||
rr_weight |
If set to priorities, then instead of sending rr_min_io requests to a path before callingpath_selector to choose the next path, the number of requests to send is determined byrr_min_io times the path's priority, as determined by the prio function. If set touniform, all path weights are equal. |
|||||||||
flush_on_last_del |
If set to yes, then multipath will disable queueing when the last path to a device has been deleted. |
multipath与oracle ASM结合
https://www.cnblogs.com/jyzhao/p/7208620.html
已有配置文件:
defaults { polling_interval 30 failback immediate no_path_retry 5 rr_min_io 100 path_checker tur user_friendly_names yes } devnode_blacklist { devnode "sda" } devices { device { vendor "IBM" product "2145" path_grouping_policy group_by_prio prio_callout "/sbin/mpath_prio_alua /dev/%n" } device { vendor "IBM" product "1750500" path_grouping_policy group_by_prio prio_callout "/sbin/mpath_prio_alua /dev/%n" } device { vendor "IBM" product "2107900" path_grouping_policy group_by_serial } device { vendor "IBM" product "2105800" path_grouping_policy group_by_serial } } multipaths { multipath { wwid 36005076801820709e000000000000043 alias dm-vote01 } }