tgtadm+多路径配置

 

环境：
centos 6.9
iSCSI Target(双网卡)：

192.168.56.150 + scsi-target-utils

192.168.56.152 + scsi-target-utils

 

iSCSI Initiator:
192.168.56.151 + iscsi-initiator-utils

 

一.服务器端配置
1. 准备要共享出去的设备，我们这里将/dev/sdb共享出去

[root@tgtamd-server ~]# fdisk -l

Disk /dev/sda: 37.6 GB, 37580963840 bytes
255 heads, 63 sectors/track, 4568 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0009da7b

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          64      512000   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2              64         587     4194304   82  Linux swap / Solaris
Partition 2 does not end on cylinder boundary.
/dev/sda3             587        4569    31992832   8e  Linux LVM

Disk /dev/sdb: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/mapper/vgroot-lvroot: 32.8 GB, 32757514240 bytes
255 heads, 63 sectors/track, 3982 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

2.服务器配置双网卡

[root@tgtamd-server ~]# ifconfig -a
eth0      Link encap:Ethernet  HWaddr 08:00:27:5D:F4:2F  
          inet addr:192.168.56.150  Bcast:192.168.56.255  Mask:255.255.255.0
          inet6 addr: fe80::a00:27ff:fe5d:f42f/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:14420 errors:0 dropped:0 overruns:0 frame:0
          TX packets:12309 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1304250 (1.2 MiB)  TX bytes:11292299 (10.7 MiB)

eth1      Link encap:Ethernet  HWaddr 08:00:27:97:F2:0F  
          inet addr:192.168.56.152  Bcast:192.168.56.255  Mask:255.255.255.0
          inet6 addr: fe80::a00:27ff:fe97:f20f/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:85 errors:0 dropped:0 overruns:0 frame:0
          TX packets:15 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:9673 (9.4 KiB)  TX bytes:1058 (1.0 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:12 errors:0 dropped:0 overruns:0 frame:0
          TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:872 (872.0 b)  TX bytes:872 (872.0 b)

3.安装SCSI服务端

[root@localhost ~]# yum install -y scsi-target-utils

4.启动服务并查看

[root@tgtamd-server ios]# service tgtd start
Starting SCSI target daemon: [ OK ]

[root@tgtamd-server ios]# chkconfig tgtd on

[root@tgtamd-server ios]# netstat -tnlp | grep :3260
tcp 0 0 0.0.0.0:3260 0.0.0.0:* LISTEN 2418/tgtd
tcp 0 0 :::3260 :::* LISTEN 2418/tgtd

5.新建一个target并查看

[root@tgtamd-server ios]# tgtadm -L iscsi -o new -m target -t 1 -T iqn.hxl.com.test

[root@tgtamd-server ios]# tgtadm -L iscsi -o show -m target
Target 1: iqn.hxl.com.test
    System information:
        Driver: iscsi
        State: ready
    I_T nexus information:
    LUN information:
        LUN: 0
            Type: controller
            SCSI ID: IET     00010000
            SCSI SN: beaf10
            Size: 0 MB, Block size: 1
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            Backing store type: null
            Backing store path: None
            Backing store flags: 
    Account information:
    ACL information:

然后必须向target中添加lun并关联存储设备才能被客户端访问使用;

6.创建一个lun

[root@tgtamd-server ios]# tgtadm -L iscsi -o new -m logicalunit -t 1 -l 1 -b /dev/sdb
[root@tgtamd-server ios]# tgtadm -L iscsi -o show -m target
Target 1: iqn.hxl.com.test
    System information:
        Driver: iscsi
        State: ready
    I_T nexus information:
    LUN information:
        LUN: 0
            Type: controller
            SCSI ID: IET     00010000
            SCSI SN: beaf10
            Size: 0 MB, Block size: 1
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            Backing store type: null
            Backing store path: None
            Backing store flags: 
        LUN: 1
            Type: disk
            SCSI ID: IET     00010001
            SCSI SN: beaf11
            Size: 4295 MB, Block size: 512
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            Backing store type: rdwr
            Backing store path: /dev/sdb
            Backing store flags:
    Account information:
    ACL information:

lun0是一个controller,而lun1则是一个磁盘;

7.开放给192.168.56.0/24网段内的主机访问

[root@tgtamd-server ios]# tgtadm -L iscsi -o bind -m target -t 1 -I 192.168.56.0/24
[root@tgtamd-server ios]# tgtadm -L iscsi -o show -m target
Target 1: iqn.hxl.com.test
    System information:
        Driver: iscsi
        State: ready
    I_T nexus information:
    LUN information:
        LUN: 0
            Type: controller
            SCSI ID: IET     00010000
            SCSI SN: beaf10
            Size: 0 MB, Block size: 1
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            Backing store type: null
            Backing store path: None
            Backing store flags: 
        LUN: 1
            Type: disk
            SCSI ID: IET     00010001
            SCSI SN: beaf11
            Size: 4295 MB, Block size: 512
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            Backing store type: rdwr
            Backing store path: /dev/sdb
            Backing store flags: 
    Account information:
    ACL information:
        192.168.56.0/24

8.写到配置文件，使重启后生效

将设置保存到配置文件 tgt-admin --dump > /etc/tgt/targets.conf

查看配置 tgt-admin -dump

二.客户端配置

1.安装SCSI客户端
[root@node2 ~]# yum install -y iscsi-initiator-utils

2.初始化，为每个initiator命名并放在/etc/iscsi/下
2.1可以自动生成
[root@tgtamd-client mnt]# echo "InitiatorName=`iscsi-iname -p iqn.hxl.com.test`" > /etc/iscsi/initiatorname.iscsi

[root@tgtamd-client mnt]# cat /etc/iscsi/initiatorname.iscsi
InitiatorName=iqn.hxl.com.test:686d0a1af5b

2.2或者为了方便记忆自定义别名
[root@tgtamd-client mnt]# echo "InitiatorAlias=initiator1" >> /etc/iscsi/initiatorname.iscsi

[root@tgtamd-client mnt]# cat /etc/iscsi/initiatorname.iscsi
InitiatorName=iqn.hxl.com.test:686d0a1af5b
InitiatorAlias=initiator1

3.启动服务
[root@tgtamd-client mnt]# service iscsi start
[root@tgtamd-client mnt]# service iscsid start
[root@tgtamd-client mnt]# chkconfig iscsi on
[root@tgtamd-client mnt]# chkconfig iscsid on

 

4.查找Target
关闭防火墙(服务端和客户端都需要关闭)
[root@tgtamd-client mnt]# service iptables stop
[root@tgtamd-client mnt]# chkconfig iptables off

5.发现网络存储

服务器网卡1

[root@tgtamd-client mnt]# iscsiadm -m discovery -t st -p 192.168.56.150
192.168.56.150:3260,1 iqn.hxl.com.test

服务器网卡2

[root@tgtamd-client mnt]# iscsiadm -m discovery -t st -p 192.168.56.152
192.168.56.150:3260,1 iqn.hxl.com.test

6. 登录服务器端使用其共享的磁盘(分区)

[root@tgtamd-client mnt]# iscsiadm -m node -T iqn.hxl.com.test -p 192.168.56.150 -l
[root@tgtamd-client mnt]# iscsiadm -m node -T iqn.hxl.com.test -p 192.168.56.152 -l

这个时候fdisk会发现2个磁盘

Disk /dev/sdc: 4294 MB, 4294967296 bytes
133 heads, 62 sectors/track, 1017 cylinders
Units = cylinders of 8246 * 512 = 4221952 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/sdb: 4294 MB, 4294967296 bytes
133 heads, 62 sectors/track, 1017 cylinders
Units = cylinders of 8246 * 512 = 4221952 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

---------------------------------配置多路径----------------------------------------------

Centos 6下配置multipath
1.安装multipath工具
# yum install device-mapper device-mapper-multipath

2、设置开机自启动
# chkconfig multipathd on

3、开启multipath
# mpathconf --enable
或者使用完整命令：mpathconf --enable --find_multipaths y --with_multipathd y --with_chkconfig y
这个时候会生成配置文件
[root@tgtamd-client yum.repos.d]# more /etc/multipath.conf

修改配置不让sda使用多路径

blacklist {
devnode "^sda"
}

4、开启服务
# service multipathd restart

5、查看状态
[root@tgtamd-client multipath]# multipath -ll
mpathb (1IET 00010001) dm-1 IET,VIRTUAL-DISK
size=4.0G features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=1 status=active
| `- 4:0:0:1 sdc 8:32 active ready running
`-+- policy='round-robin 0' prio=1 status=enabled
`- 3:0:0:1 sdb 8:16 active ready running

6.查看mapper下面的设备情况
[root@tgtamd-client multipath]# ls -al /dev/mapper/
total 0
drwxr-xr-x. 2 root root 100 Dec 23 10:49 .
drwxr-xr-x. 21 root root 3840 Dec 23 10:49 ..
crw-rw----. 1 root root 10, 58 Dec 23 10:30 control
lrwxrwxrwx. 1 root root 7 Dec 23 10:52 mpathb -> ../dm-1
lrwxrwxrwx. 1 root root 7 Dec 23 10:30 vgroot-lvroot -> ../dm-0

7.修改设备别名
查看wwwid
[root@tgtamd-client multipath]# more /etc/multipath/wwids
# Multipath wwids, Version : 1.0
# NOTE: This file is automatically maintained by multipath and multipathd.
# You should not need to edit this file in normal circumstances.
#
# Valid WWIDs:
/1IET 00010001/

或是使用scsi_id查看
[root@tgtamd-client multipath]# /lib/udev/scsi_id --whitelisted --device=/dev/sdb
1IET 00010001
[root@tgtamd-client multipath]# /lib/udev/scsi_id --whitelisted --device=/dev/sdc
1IET 00010001

这里/dev/sdb和/dev/sdc是多路径产生的

修改/etc/multipath.conf文件

multipaths {
multipath {
wwid "1IET 00010001"
alias ocr01
}
}

重新启动multipath
[root@tgtamd-client multipath]# service multipathd restart

再次查看
[root@tgtamd-client multipath]# multipath -ll
ocr01 (1IET 00010001) dm-1 IET,VIRTUAL-DISK
size=4.0G features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=1 status=active
| `- 4:0:0:1 sdc 8:32 active ready running
`-+- policy='round-robin 0' prio=1 status=enabled
`- 3:0:0:1 sdb 8:16 active ready running

[root@tgtamd-client multipath]# ls -al /dev/mapper/
total 0
drwxr-xr-x. 2 root root 100 Dec 23 11:01 .
drwxr-xr-x. 21 root root 3840 Dec 23 10:49 ..
crw-rw----. 1 root root 10, 58 Dec 23 10:30 control
lrwxrwxrwx. 1 root root 7 Dec 23 11:01 ocr01 -> ../dm-1
lrwxrwxrwx. 1 root root 7 Dec 23 10:30 vgroot-lvroot -> ../dm-0