随着云计算和SDN的应用越来越多,对于SDN控制器的选择也有许多,例如OpenDaylight、RYU、Floodlight和POX等,我们先选择RYU与openstack集成作为研究方向。RYU作为Python开发的轻量级SDN控制器,可以非常容易的可以与Openstack集成在一起,现在Openstack集成SDN的方案中使用RYU的项目主要包含以下几种:
方案1. openstack在Icehouse和Juno版中neutron已经包含RyuNeutronPluginV2作为核心插件实现SDN控制器。
方案2. openstack在Kilo和Liberty版中ryu的插件已经从neutron中移除,而是通过ofagent机制driver来实现RYU控制器的集成
方案3. Dragonflow项目作为一个轻量级集成RYU控制器的SDN项目,实现了L2和分布式L3等功能。
以上三个基于RYU控制器的sdn解决方案各有特点:
方案1是配置RyuNeutronPluginV2插件,但是如果使用此插件,就无法使用ml2插件,也就无法同时支持多type和mech驱动,方案的可扩展性大大降低,同时新版本openstack也不再使用此解决方案。
方案2是基于ml2插件基础上,配置ofagent机制驱动来实现RYU的集成,使用ml2插件就可以很好的支持多种type和mech驱动,因此在新版本openstack已经开始推荐使用ofagent机制驱动方式。
方案3是以色列华为团队独立开发的基于ryu的sdn控制器项目,它实现了L2、分布L3、arp和dhcp等多种网络功能,可以说是基于RYU的SDN方案中比较有特色和有创新的方案。
作为研究RYU控制器与openstack集成的目的,此三种方案都有必要进行一下学习和研究。我将分三种方案的平台分别进行研究,整理出各自方案的特点。
1.OFAgent方案介绍
OFAgent是一个neutron的核心插件,实现了ML2的机制驱动。它的目标是支持一个纯Openflow1.3协议的交换机。
OFAgent控制器与openstack集成需要在控制节点(网络节点)和计算节点部署OFAgent插件agent,来控制br-int集成网桥,为了快速部署OFAgent试验环境,可以采用devstack来部署ofagent插件环境,以下为控制节点和计算节点部署配置参考
https://wiki.openstack.org/wiki/Neutron/OFAgent
http://docs.openstack.org/developer/devstack/guides/neutron.html
2.OFAgent方案部署情况说明
private网络 private-subnet 10.0.0.0/24 fd5b:1720:acf7:0:f816:3eff:fe95:7b54
public 网络public-subnet 172.24.4.0/24 2001:db8::1
port信息:
36515a5c-737c-4045-8efb-9b43e3a67957 fa:16:3e:95:7b:54 10.0.0.2
bd8b208d-2d20-4bc2-a152-8d38b63ae106 fa:16:3e:d4:aa:a4 10.0.0.1
b6deaa40-290f-4cf4-baec-52c5305368c0 fa:16:3e:6e:97:bf fd5b:1720:acf7::1
55b2e0e0-b9bc-44d8-bae4-45ed3eb4a60f fa:16:3e:8a:a2:40 172.24.4.2
未创建虚机是网络配置如下:
1)控制节点
port2为(tap36515a5c-73): addr:92:af:57:37:9c:d6
port3为(tapb6deaa40-29): addr:5e:57:c4:1e:d1:7f
port4为(tapbd8b208d-2d): addr:a6:f2:6f:e1:d5:55
port5为(_ofa-tun-vxlan): addr:a2:40:6a:f6:5e:cc
[stack@controller devstack]$ sudo ovs-vsctl show cb21984d-ee33-4ac7-8e03-0cfb57010b3e Bridge br-int Controller "tcp:127.0.0.1:6633" is_connected: true Port br-int Interface br-int type: internal Port veth-pub-int Interface veth-pub-int error: "could not open network device veth-pub-int (No such device)" Port "tap36515a5c-73" Interface "tap36515a5c-73" Port "tapbd8b208d-2d" Interface "tapbd8b208d-2d" Port _ofa-tun-vxlan Interface _ofa-tun-vxlan type: vxlan options: {df_default="true", in_key=flow, local_ip="0", out_key=flow, remote_ip=flow} Port "tapb6deaa40-29" Interface "tapb6deaa40-29" ovs_version: "2.4.0"
[stack@controller devstack]$ sudo ovs-ofctl dump-flows br-int -O openflow13 OFPST_FLOW reply (OF1.3) (xid=0x2): cookie=0x0, duration=866.458s, table=0, n_packets=14, n_bytes=1652, priority=1,in_port=3 actions=write_metadata:0x10001/0x10fff,goto_table:4 cookie=0x0, duration=866.023s, table=0, n_packets=0, n_bytes=0, priority=1,in_port=2 actions=write_metadata:0x10001/0x10fff,goto_table:4 cookie=0x0, duration=865.610s, table=0, n_packets=0, n_bytes=0, priority=1,in_port=4 actions=write_metadata:0x10001/0x10fff,goto_table:4 cookie=0x0, duration=866.907s, table=0, n_packets=0, n_bytes=0, priority=0 actions=drop cookie=0x0, duration=866.905s, table=1, n_packets=0, n_bytes=0, priority=0 actions=drop cookie=0x0, duration=866.460s, table=2, n_packets=0, n_bytes=0, priority=1,tun_id=0x42b actions=write_metadata:0x1/0xfff,goto_table:7 cookie=0x0, duration=866.903s, table=2, n_packets=0, n_bytes=0, priority=0 actions=drop cookie=0x0, duration=866.901s, table=3, n_packets=0, n_bytes=0, priority=0 actions=goto_table:7 cookie=0x0, duration=866.899s, table=4, n_packets=14, n_bytes=1652, priority=0 actions=goto_table:5 cookie=0x0, duration=866.897s, table=5, n_packets=14, n_bytes=1652, priority=0 actions=goto_table:6 cookie=0x0, duration=866.895s, table=6, n_packets=0, n_bytes=0, priority=1,arp,arp_op=1 actions=CONTROLLER:65509 cookie=0x0, duration=866.893s, table=6, n_packets=14, n_bytes=1652, priority=0 actions=goto_table:7 cookie=0x0, duration=866.890s, table=7, n_packets=14, n_bytes=1652, priority=0 actions=goto_table:8 cookie=0x0, duration=866.452s, table=8, n_packets=0, n_bytes=0, priority=1,metadata=0x1/0xfff,dl_dst=fa:16:3e:6e:97:bf actions=output:3 cookie=0x0, duration=866.016s, table=8, n_packets=0, n_bytes=0, priority=1,metadata=0x1/0xfff,dl_dst=fa:16:3e:95:7b:54 actions=output:2 cookie=0x0, duration=865.603s, table=8, n_packets=0, n_bytes=0, priority=1,metadata=0x1/0xfff,dl_dst=fa:16:3e:d4:aa:a4 actions=output:4 cookie=0x0, duration=866.888s, table=8, n_packets=14, n_bytes=1652, priority=0 actions=goto_table:9 cookie=0x0, duration=866.886s, table=9, n_packets=14, n_bytes=1652, priority=0 actions=goto_table:10 cookie=0x0, duration=866.885s, table=10, n_packets=14, n_bytes=1652, priority=0 actions=goto_table:11 cookie=0x0, duration=866.883s, table=11, n_packets=14, n_bytes=1652, priority=0 actions=goto_table:12 cookie=0x0, duration=866.881s, table=12, n_packets=14, n_bytes=1652, priority=0 actions=goto_table:13 cookie=0x0, duration=865.607s, table=13, n_packets=14, n_bytes=1652, priority=1,metadata=0x1/0xfff,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=output:3,output:2,output:4 cookie=0x0, duration=866.880s, table=13, n_packets=0, n_bytes=0, priority=0 actions=drop
2)计算节点
port2为 2(_ofa-tun-vxlan): addr:9e:aa:01:13:76:6f vxlan隧道端口连接网络节点
[stack@compute1 ~]$ sudo ovs-vsctl show 71ebef35-168e-4726-971a-b5f6184d89d1 Bridge br-int Controller "tcp:127.0.0.1:6633" is_connected: true Port br-int Interface br-int type: internal Port _ofa-tun-vxlan Interface _ofa-tun-vxlan type: vxlan options: {df_default="true", in_key=flow, local_ip="0", out_key=flow, remote_ip=flow} Port veth-pub-int Interface veth-pub-int error: "could not open network device veth-pub-int (No such device)"
[stack@compute1 ~]$ sudo ovs-ofctl dump-flows br-int -O openflow13 OFPST_FLOW reply (OF1.3) (xid=0x2): cookie=0x0, duration=21806.716s, table=0, n_packets=0, n_bytes=0, priority=1,tun_dst=10.255.255.101,in_port=2 actions=goto_table:2 cookie=0x0, duration=21868.188s, table=0, n_packets=6, n_bytes=468, priority=0 actions=drop cookie=0x0, duration=21868.186s, table=1, n_packets=0, n_bytes=0, priority=0 actions=drop cookie=0x0, duration=21868.184s, table=2, n_packets=0, n_bytes=0, priority=0 actions=drop cookie=0x0, duration=21868.183s, table=3, n_packets=0, n_bytes=0, priority=0 actions=goto_table:7 cookie=0x0, duration=21868.181s, table=4, n_packets=0, n_bytes=0, priority=0 actions=goto_table:5 cookie=0x0, duration=21868.180s, table=5, n_packets=0, n_bytes=0, priority=0 actions=goto_table:6 cookie=0x0, duration=21868.178s, table=6, n_packets=0, n_bytes=0, priority=1,arp,arp_op=1 actions=CONTROLLER:65509 cookie=0x0, duration=21868.176s, table=6, n_packets=0, n_bytes=0, priority=0 actions=goto_table:7 cookie=0x0, duration=21789.918s, table=7, n_packets=0, n_bytes=0, priority=1,metadata=0x10001/0x10fff,dl_dst=fa:16:3e:95:7b:54 actions=set_field:0x42b->tun_id,set_field:10.255.255.100->tun_dst,output:2 cookie=0x0, duration=21789.117s, table=7, n_packets=0, n_bytes=0, priority=1,metadata=0x10001/0x10fff,dl_dst=fa:16:3e:d4:aa:a4 actions=set_field:0x42b->tun_id,set_field:10.255.255.100->tun_dst,output:2 cookie=0x0, duration=21788.094s, table=7, n_packets=0, n_bytes=0, priority=1,metadata=0x10001/0x10fff,dl_dst=fa:16:3e:6e:97:bf actions=set_field:0x42b->tun_id,set_field:10.255.255.100->tun_dst,output:2 cookie=0x0, duration=21868.175s, table=7, n_packets=0, n_bytes=0, priority=0 actions=goto_table:8 cookie=0x0, duration=21868.173s, table=8, n_packets=0, n_bytes=0, priority=0 actions=goto_table:9 cookie=0x0, duration=21868.170s, table=9, n_packets=0, n_bytes=0, priority=0 actions=goto_table:10 cookie=0x0, duration=21868.169s, table=10, n_packets=0, n_bytes=0, priority=0 actions=goto_table:11 cookie=0x0, duration=21806.714s, table=11, n_packets=0, n_bytes=0, priority=1,metadata=0x10001/0x10fff actions=set_field:0x42b->tun_id,set_field:10.255.255.100->tun_dst,output:2,goto_table:12 cookie=0x0, duration=21868.167s, table=11, n_packets=0, n_bytes=0, priority=0 actions=goto_table:12 cookie=0x0, duration=21868.166s, table=12, n_packets=0, n_bytes=0, priority=0 actions=goto_table:13 cookie=0x0, duration=21807.120s, table=13, n_packets=0, n_bytes=0, priority=1,metadata=0x1/0xfff,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=output:1 cookie=0x0, duration=21868.164s, table=13, n_packets=0, n_bytes=0, priority=0 actions=drop
在计算节点compute1上创建testvm1虚机
#nova list
| 88873f79-39f2-4ee9-bf05-4967980fc640 | testvm1 | ACTIVE | - | Running | private=fd5b:1720:acf7:0:f816:3eff:fe1e:e11c, 10.0.0.13 |
#neutron port-list
cfcb5daf-38e4-4d39-a718-4d3ed713b423 | | fa:16:3e:1e:e1:1c | {"subnet_id": "0626d00a-d74c-447f-ac4e-e55ce3b68bce", |
| | | | "ip_address": "10.0.0.13"}
网络节点br-int流表
[stack@controller devstack]$ sudo ovs-ofctl dump-flows br-int -O openflow13 OFPST_FLOW reply (OF1.3) (xid=0x2): cookie=0x0, duration=482.694s, table=0, n_packets=13, n_bytes=1534, priority=1,in_port=2 actions=write_metadata:0x10001/0x10fff,goto_table:4 cookie=0x0, duration=482.288s, table=0, n_packets=11, n_bytes=1456, priority=1,in_port=1 actions=write_metadata:0x10001/0x10fff,goto_table:4 cookie=0x0, duration=481.645s, table=0, n_packets=2, n_bytes=140, priority=1,in_port=3 actions=write_metadata:0x10001/0x10fff,goto_table:4 cookie=0x0, duration=285.849s, table=0, n_packets=16, n_bytes=1832, priority=1,tun_dst=10.255.255.100,in_port=5 actions=goto_table:2 cookie=0x0, duration=544.103s, table=0, n_packets=26, n_bytes=2448, priority=0 actions=drop cookie=0x0, duration=544.101s, table=1, n_packets=0, n_bytes=0, priority=0 actions=drop cookie=0x0, duration=482.696s, table=2, n_packets=16, n_bytes=1832, priority=1,tun_id=0x42b actions=write_metadata:0x1/0xfff,goto_table:7 cookie=0x0, duration=544.096s, table=2, n_packets=0, n_bytes=0, priority=0 actions=drop cookie=0x0, duration=544.094s, table=3, n_packets=0, n_bytes=0, priority=0 actions=goto_table:7 cookie=0x0, duration=544.092s, table=4, n_packets=26, n_bytes=3130, priority=0 actions=goto_table:5 cookie=0x0, duration=544.091s, table=5, n_packets=26, n_bytes=3130, priority=0 actions=goto_table:6 cookie=0x0, duration=544.089s, table=6, n_packets=2, n_bytes=84, priority=1,arp,arp_op=1 actions=CONTROLLER:65509 cookie=0x0, duration=544.087s, table=6, n_packets=24, n_bytes=3046, priority=0 actions=goto_table:7 cookie=0x0, duration=285.841s, table=7, n_packets=9, n_bytes=1364, priority=1,metadata=0x10001/0x10fff,dl_dst=fa:16:3e:1e:e1:1c actions=set_field:0x42b->tun_id,set_field:10.255.255.101->tun_dst,output:5 cookie=0x0, duration=544.086s, table=7, n_packets=31, n_bytes=3514, priority=0 actions=goto_table:8 cookie=0x0, duration=482.688s, table=8, n_packets=0, n_bytes=0, priority=1,metadata=0x1/0xfff,dl_dst=fa:16:3e:6e:97:bf actions=output:2 cookie=0x0, duration=482.281s, table=8, n_packets=6, n_bytes=510, priority=1,metadata=0x1/0xfff,dl_dst=fa:16:3e:95:7b:54 actions=output:1 cookie=0x0, duration=481.639s, table=8, n_packets=1, n_bytes=98, priority=1,metadata=0x1/0xfff,dl_dst=fa:16:3e:d4:aa:a4 actions=output:3 cookie=0x0, duration=544.084s, table=8, n_packets=24, n_bytes=2906, priority=0 actions=goto_table:9 cookie=0x0, duration=544.082s, table=9, n_packets=24, n_bytes=2906, priority=0 actions=goto_table:10 cookie=0x0, duration=544.080s, table=10, n_packets=24, n_bytes=2906, priority=0 actions=goto_table:11 cookie=0x0, duration=285.847s, table=11, n_packets=7, n_bytes=826, priority=1,metadata=0x10001/0x10fff actions=set_field:0x42b->tun_id,set_field:10.255.255.101->tun_dst,output:5,goto_table:12 cookie=0x0, duration=543.867s, table=11, n_packets=17, n_bytes=2080, priority=0 actions=goto_table:12 cookie=0x0, duration=543.866s, table=12, n_packets=24, n_bytes=2906, priority=0 actions=goto_table:13 cookie=0x0, duration=481.643s, table=13, n_packets=24, n_bytes=2906, priority=1,metadata=0x1/0xfff,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=output:2,output:1,output:3 cookie=0x0, duration=543.864s, table=13, n_packets=0, n_bytes=0, priority=0 actions=drop
计算节点流表:
port1为 1(qvocfcb5daf-38): addr:0e:f9:4b:69:bf:d6 testvm1虚机连接br-int端口
port2为 2(_ofa-tun-vxlan): addr:9e:aa:01:13:76:6f vxlan隧道端口连接网络节点
[stack@compute1 devstack]$ sudo ovs-ofctl dump-flows br-int -O openflow13 OFPST_FLOW reply (OF1.3) (xid=0x2): cookie=0x0, duration=324.285s, table=0, n_packets=18, n_bytes=1916, priority=1,in_port=1 actions=write_metadata:0x10001/0x10fff,goto_table:4 cookie=0x0, duration=323.856s, table=0, n_packets=17, n_bytes=2308, priority=1,tun_dst=10.255.255.101,in_port=2 actions=goto_table:2 cookie=0x0, duration=600.046s, table=0, n_packets=3, n_bytes=250, priority=0 actions=drop cookie=0x0, duration=600.043s, table=1, n_packets=0, n_bytes=0, priority=0 actions=drop cookie=0x0, duration=324.288s, table=2, n_packets=17, n_bytes=2308, priority=1,tun_id=0x42b actions=write_metadata:0x1/0xfff,goto_table:7 cookie=0x0, duration=600.040s, table=2, n_packets=0, n_bytes=0, priority=0 actions=drop cookie=0x0, duration=600.038s, table=3, n_packets=0, n_bytes=0, priority=0 actions=goto_table:7 cookie=0x0, duration=600.036s, table=4, n_packets=18, n_bytes=1916, priority=0 actions=goto_table:5 cookie=0x0, duration=600.034s, table=5, n_packets=18, n_bytes=1916, priority=0 actions=goto_table:6 cookie=0x0, duration=600.033s, table=6, n_packets=2, n_bytes=84, priority=1,arp,arp_op=1 actions=CONTROLLER:65509 cookie=0x0, duration=600.031s, table=6, n_packets=16, n_bytes=1832, priority=0 actions=goto_table:7 cookie=0x0, duration=323.847s, table=7, n_packets=6, n_bytes=510, priority=1,metadata=0x10001/0x10fff,dl_dst=fa:16:3e:95:7b:54 actions=set_field:0x42b->tun_id,set_field:10.255.255.100->tun_dst,output:2 cookie=0x0, duration=323.843s, table=7, n_packets=0, n_bytes=0, priority=1,metadata=0x10001/0x10fff,dl_dst=fa:16:3e:6e:97:bf actions=set_field:0x42b->tun_id,set_field:10.255.255.100->tun_dst,output:2 cookie=0x0, duration=323.841s, table=7, n_packets=1, n_bytes=98, priority=1,metadata=0x10001/0x10fff,dl_dst=fa:16:3e:d4:aa:a4 actions=set_field:0x42b->tun_id,set_field:10.255.255.100->tun_dst,output:2 cookie=0x0, duration=600.029s, table=7, n_packets=26, n_bytes=3532, priority=0 actions=goto_table:8 cookie=0x0, duration=324.272s, table=8, n_packets=9, n_bytes=1364, priority=1,metadata=0x1/0xfff,dl_dst=fa:16:3e:1e:e1:1c actions=output:1 cookie=0x0, duration=600.027s, table=8, n_packets=17, n_bytes=2168, priority=0 actions=goto_table:9 cookie=0x0, duration=600.026s, table=9, n_packets=17, n_bytes=2168, priority=0 actions=goto_table:10 cookie=0x0, duration=600.024s, table=10, n_packets=17, n_bytes=2168, priority=0 actions=goto_table:11 cookie=0x0, duration=323.854s, table=11, n_packets=9, n_bytes=1224, priority=1,metadata=0x10001/0x10fff actions=set_field:0x42b->tun_id,set_field:10.255.255.100->tun_dst,output:2,goto_table:12 cookie=0x0, duration=600.023s, table=11, n_packets=8, n_bytes=944, priority=0 actions=goto_table:12 cookie=0x0, duration=600.020s, table=12, n_packets=17, n_bytes=2168, priority=0 actions=goto_table:13 cookie=0x0, duration=324.279s, table=13, n_packets=17, n_bytes=2168, priority=1,metadata=0x1/0xfff,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=output:1 cookie=0x0, duration=600.018s, table=13, n_packets=0, n_bytes=0, priority=0 actions=drop
3.实验环境OFAgent运行流表分析:
table
0 in_port检查表
1 GRE隧道进入表
2 Vxlan隧道进入表
3 vlan和flat物理网络表(虚机发出)
4 本地流入表
5 arp未知tpa(目标ip地址)转发表
6 arp应答表
7 隧道发出表
8 本地发出表(发往虚机)
9 vlan和flat物理网络表(发往虚机)
10 GRE隧道广播表
11 Vxlan隧道广播表
12 vlan和flat物理网络广播表
13 本地广播表
4.OFAgent流表设计图
根据OFAgent项目代码中flows设计文档可以总结一下流表设计规则