最近有个客户需求是在某12.2版本的RAC环境上添加心跳网络,顺便考虑将之前的心跳网络改为asm专用。我目前只有19c的RAC的测试环境(19c是12c的最终稳定版本),直接测试验证下过程备忘。
1.测试环境概述
首先我的测试环境是这样的:[grid@db195 ~]$ oifcfg getif
enp0s3 192.168.1.0 global public
enp0s8 10.10.1.0 global cluster_interconnect,asm
2.添加心跳网络
主机层面添加新的网卡enp0s9,配置对应的IP地址后,可以将enp0s9添加为心跳网络:[grid@db195 ~]$ oifcfg setif -global enp0s9/10.10.1.0:cluster_interconnect
[grid@db195 ~]$ oifcfg getif
enp0s3 192.168.1.0 global public
enp0s8 10.10.1.0 global cluster_interconnect,asm
enp0s9 10.10.1.0 global cluster_interconnect
3.修改asm网络
如果确认需要将之前的设置为asm专用,可以再修改:[grid@db195 ~]$ oifcfg setif -global enp0s8/10.10.1.0:asm
[grid@db195 ~]$ oifcfg getif
enp0s3 192.168.1.0 global public
enp0s8 10.10.1.0 global asm
enp0s9 10.10.1.0 global cluster_interconnect
此时查看RAC其他节点的配置确保一样。
4.重启has集群
使用ifconfig -a查看新增网卡enp0s9的信息,还是没有被HAIP接管的(没有对应169网段的地址信息)enp0s9: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.10.1.196 netmask 255.255.255.0 broadcast 10.10.1.255
inet6 fe80::2549:d14d:7dfd:61c4 prefixlen 64 scopeid 0x20<link>
ether 08:00:27:59:1a:a6 txqueuelen 1000 (Ethernet)
RX packets 66 bytes 19094 (18.6 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 16 bytes 1170 (1.1 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
需要使用crsctl stop has关闭各节点集群:
[root@db193 ~]# /u01/app/19.3.0/grid/bin/crsctl stop has
[root@db195 ~]# /u01/app/19.3.0/grid/bin/crsctl stop has
[root@db193 ~]# /u01/app/19.3.0/grid/bin/crsctl start has
[root@db195 ~]# /u01/app/19.3.0/grid/bin/crsctl start has
5.检查心跳状态
重启后,再次观察ifconfig -a看到新增网卡enp0s9有对应的HAIP地址了,但是enp0s8没有了,因为enp0s8只是用的asm:enp0s8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.10.1.195 netmask 255.255.255.0 broadcast 10.10.1.255
inet6 fe80::5a1e:e7de:fe1a:a286 prefixlen 64 scopeid 0x20<link>
ether 08:00:27:1d:85:d4 txqueuelen 1000 (Ethernet)
RX packets 38565 bytes 27624748 (26.3 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 69723 bytes 72018893 (68.6 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
enp0s9: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.10.1.196 netmask 255.255.255.0 broadcast 10.10.1.255
inet6 fe80::2549:d14d:7dfd:61c4 prefixlen 64 scopeid 0x20<link>
ether 08:00:27:59:1a:a6 txqueuelen 1000 (Ethernet)
RX packets 12510 bytes 7500503 (7.1 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 14723 bytes 17562072 (16.7 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
enp0s9:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 169.254.3.55 netmask 255.255.224.0 broadcast 169.254.31.255
ether 08:00:27:59:1a:a6 txqueuelen 1000 (Ethernet)
如果之前的enp0s8网卡没有修改为asm专用,也就是asm,cluster_interconnect的话,那么enp0s8网卡就会保持有HAIP地址:
--设置enp0s8为asm,cluster_interconnect
[grid@db195 ~]$ oifcfg setif -global enp0s8/10.10.1.0:asm,cluster_interconnect
--重启集群后,ifconfig -a就会发现enp0s8网卡保持有HAIP地址:
enp0s8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.10.1.195 netmask 255.255.255.0 broadcast 10.10.1.255
inet6 fe80::5a1e:e7de:fe1a:a286 prefixlen 64 scopeid 0x20<link>
ether 08:00:27:1d:85:d4 txqueuelen 1000 (Ethernet)
RX packets 41817 bytes 28559675 (27.2 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 89166 bytes 88323795 (84.2 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
enp0s8:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 169.254.7.148 netmask 255.255.240.0 broadcast 169.254.15.255
ether 08:00:27:1d:85:d4 txqueuelen 1000 (Ethernet)
enp0s9: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.10.1.196 netmask 255.255.255.0 broadcast 10.10.1.255
inet6 fe80::2549:d14d:7dfd:61c4 prefixlen 64 scopeid 0x20<link>
ether 08:00:27:59:1a:a6 txqueuelen 1000 (Ethernet)
RX packets 103509 bytes 77082525 (73.5 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 237436 bytes 290562629 (277.1 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
enp0s9:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 169.254.22.254 netmask 255.255.240.0 broadcast 169.254.31.255
ether 08:00:27:59:1a:a6 txqueuelen 1000 (Ethernet)
注意:如果新增enp0s9心跳网卡和之前心跳网段不一样,比如10.10.2.195,还要注意和oifcfg setif enp0s9/10.10.2.0:cluster_interconnect,网段与新网卡的网路保持一致,相互匹配。
总结下这个事情就是先确认物理增加心跳网卡和心跳线,然后数据库层面直接使用oifcfg可增加或修改,需要重启has集群生效。