• 打赏 jQuery火箭图标返回顶部代码</h1> <hr> <span class="pre-scrollable"><div id="cnblogs_post_body" class="blogpost-body"> <div class="DocReader-module_content_2JTe8"> <div class="index-module_content_Sqtvu"> <div class="yuque-doc-content" data-df="lake"> <div class="lake-engine-view lake-typography-traditional"> <p data-lake-id="3da1156c5fbd9857adaef66dfe0fc3c8_p_0" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">Kubernetes中解决网络跨主机通信的一个经典插件就是Flannel。Flannel实质上只是一个框架,真正为我们提供网络功能的是后端的Flannel实现,目前Flannel后端实现的方式有三种:</span></p> <ol start="1" data-lake-id="b5e00391caea190b040b930f85043082_ol_0"> <li data-lake-id="f81d40682a780a3f64d98220c9f48c27_li_0" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">UDP</span></li> <li data-lake-id="bf96bd6f0e39f8986c3b794808b3a6a3_li_1" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">VXLAN</span></li> <li data-lake-id="7cd38b585c3ea2206341e9644de6d579_li_2" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">HOST-GW</span></li> </ol> <h2 id="9xYPB" data-lake-id="9abce1f60f58c80cb234a282c0813299_h1_0" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">一、UDP</span></h2> <p data-lake-id="5613dd09f32f9da42c9b7a1f84cdcff8_p_2" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">UDP是最早的实现方式,但是由于其性能原因,现已经被废弃,但是UDP模式是最直接,也最容易理解的跨主机实现方式。</span></p> <p data-lake-id="1ec90f75e8c8ab400688cc641c1c660e_p_4" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">假如有两台Node,如下:</span></p> <ol start="1" data-lake-id="b2851bd9b0f49093eae7aaaac7566fe5_ol_1"> <li data-lake-id="a4d14eea1e8ec9e51a6b6848f5ed875c_li_3" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">Node01上有容器nginx01,其IP为172.20.1.107,其docker0的地址为172.20.1.1/24;</span></li> <li data-lake-id="c292a159e32a705fc95fbc5a20a3c553_li_4" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">Node02上有容器nginx02,其IP为172.20.2.133,其docker0的地址为172.20.2.1/24;</span></li> </ol> <p data-lake-id="ccbacff586f99973f39f8f08173cffca_p_6" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">那么现在nginx01要访问nginx02,其流程应该是怎么样的呢?</span></p> <ol start="1" data-lake-id="b7d87c62bca4d8306c0f88db9da4f13a_ol_2"> <li data-lake-id="b6b0fe6b34ed354c77ce824df1162c13_li_5" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">首先从nginx01发送IP包,源IP是172.20.1.107,目的IP是172.20.2.133。</span></li> <li data-lake-id="add77a2ce22f1edd6b449a0005c78c8c_li_6" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">由于目的IP并不在Node01上的docker0网桥里,所以会将包通过默认路由转发到docker0网桥所在的宿主机上;</span></li> <li data-lake-id="a9cc8e11c0d3bf247ad9f6bfbc10d8ce_li_7" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">它会通过本地的路由规则,转发到下一个目的IP,我们可以通过ip route查看本地的路由信息,通过路由信息可以看到它被转发到一个flannel0的设备中;</span></li> <li data-lake-id="0d4ea8f02665586007db853630b15b23_li_8" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">flannel0设备会把这个IP包交给创建这个设备的应用程序,也就是Flannel进程(从内核状态向用户状态切换);</span></li> <li data-lake-id="a0a1148f6083db1e793b261eb9b48371_li_9" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">Flannel进程收到IP包后,将这个包封装在UDP中,就根据其目的地址将其转发给Node02(通过每个宿主机上监听的8285端口),这时候的源地址是Node01的地址,目的地址是Node02的地址;</span></li> <li data-lake-id="2a5207e222652237b36cfd3fc1615eed_li_10" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">Node02收到包后,就会直接将其转发给flannel0设备,然后进行解包,匹配本地路由规则转发给docker0网桥,然后docker0网桥就扮演二层交换机的功能,将包转发到最终的目的地;</span></li> </ol> <p data-lake-id="96bc672a482439ea7ef405f0096fda69_p_8" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">其流程图如下:</span></p> <p data-lake-id="7aceb1cd393868b8ffaf06aed880d774_p_9"><span style="font-family: "Microsoft YaHei"; font-size: 14px" data-card-type="inline" data-lake-card="image" data-card-value="data:%7B%22src%22%3A%22https%3A%2F%2Fcdn.nlark.com%2Fyuque%2F0%2F2019%2Fpng%2F255717%2F1572503803035-05b3a4b3-6dbd-4ce2-9ccd-dced0d111a66.png%22%2C%22originWidth%22%3A824%2C%22originHeight%22%3A467%2C%22name%22%3A%22image.png%22%2C%22size%22%3A53368%2C%22display%22%3A%22inline%22%2C%22align%22%3A%22left%22%2C%22linkTarget%22%3A%22_blank%22%2C%22status%22%3A%22done%22%2C%22style%22%3A%22none%22%2C%22search%22%3A%22%22%2C%22margin%22%3A%7B%22top%22%3Afalse%2C%22bottom%22%3Afalse%7D%2C%22width%22%3A824%2C%22height%22%3A467%7D"><img src="https://cdn.nlark.com/yuque/0/2019/png/255717/1572503803035-05b3a4b3-6dbd-4ce2-9ccd-dced0d111a66.png" alt="image.png" class="image lake-drag-image" title="image.png" data-role="image" data-raw-src="" data-height="467px"></span></p> <p data-lake-id="f7c2fb43f3f9a318725a841f54596183_p_10" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">注:</span></p> <p data-lake-id="1ccb930f1ff76547e48dc2332b023d1a_p_11" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">1、flannel0是一个TUN设备,它的作用是在操作系统和应用程序之间传递IP包;</span></p> <p data-lake-id="7fcdad8709f2b6176a160b2cc38a5887_p_12" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">2、Flannel是根据子网(Subnet)来查看IP地址对应的容器是运行在那个Node上;</span></p> <p data-lake-id="6cd9e2d4cc7f69dce66fde3886cac907_p_13" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">3、这些子网和Node的对应关系,是保存在Etcd中(仅限UDP模式);</span></p> <p data-lake-id="de8d8ea4286859bd4f528db0f93e6678_p_14" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">4、UDP模式其实是一个三层的Overlay网络;它首先对发出的IP包进行UDP封装,然后接收端对包进行解封拿到原始IP,进而把这个包转发给目标容器。这就好比Flannel在不同的宿主机上的两容器之间打通了一条隧道,使得这个两个IP可以通信,而无需关心容器和宿主机的分布情况;</span></p> <p data-lake-id="48aa602eed44ab05790beac9762cdd95_p_16" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">UDP之所以被废弃是主要是由于其仅在发包的过程中就在用户态和内核态进行来回的数据交换,这样的性能代价是很高的。如下:</span></p> <p data-lake-id="97c047a084e4bf32f2aa42faf2b6e2fe_p_17"><span style="font-family: "Microsoft YaHei"; font-size: 14px" data-card-type="inline" data-lake-card="image" data-card-value="data:%7B%22src%22%3A%22https%3A%2F%2Fcdn.nlark.com%2Fyuque%2F0%2F2019%2Fpng%2F255717%2F1572504458923-ef690370-488f-4969-9631-8e88489a19f6.png%22%2C%22originWidth%22%3A890%2C%22originHeight%22%3A593%2C%22size%22%3A0%2C%22display%22%3A%22inline%22%2C%22align%22%3A%22left%22%2C%22linkTarget%22%3A%22_blank%22%2C%22status%22%3A%22done%22%2C%22style%22%3A%22none%22%2C%22search%22%3A%22%22%2C%22margin%22%3A%7B%22top%22%3Afalse%2C%22bottom%22%3Afalse%7D%2C%22width%22%3A890%2C%22height%22%3A593%7D"><img src="https://cdn.nlark.com/yuque/0/2019/png/255717/1572504458923-ef690370-488f-4969-9631-8e88489a19f6.png" alt="image" class="image lake-drag-image" title="image" data-role="image" data-raw-src="" data-height="593px"></span></p> <h2 id="X2rWP" data-lake-id="a89b9790d8f501a8dbc32c2e6eb59788_h1_1" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">二、VXLAN</span></h2> <p data-lake-id="36f636d13a81436737f5aca6ee4fe7b6_p_19" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">VXLAN:Virtual Extensible LAN(虚拟可扩展局域网),是Linux内核本身就支持的一种虚拟化网络技术,它可以完全在内核态实现上述的封装和解封装过程,减少用户态到内核态的切换次数,把核心的处理逻辑都放到内核态,其通过与前面相似的隧道技术,构建出覆盖网络或者叠加网络(Overlay Network)。</span></p> <p data-lake-id="ad296618078d616e72f30f2ac29d95ed_p_21" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">其设计思想为在现有的三层网络下,叠加一层虚拟的并由内核VXLAN维护的二层网络,使得连接在这个二层网络上的主机可以像在局域网一样通信。</span></p> <p data-lake-id="356a93e6f2638b1945929c9b5e6171c7_p_23" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">为了能够在二层网络中打通隧道,VXLAN会在宿主机上设置一个特殊的网络设备作为隧道的两端,这个隧道就叫VTEP(Virtual Tunnel End Point 虚拟隧道端点)。而VTEP的作用跟上面的flanneld进程非常相似,只不过它进行封装和解封的对象是二层的数据帧,而且这个工作的执行流程全部在内核中完成。</span></p> <p data-lake-id="21244b185d509f43c563f6044885af28_p_24" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">其流程如下:</span></p> <p data-lake-id="5c840240d2087c981415cb06bd8447f6_p_25"><span style="font-family: "Microsoft YaHei"; font-size: 14px" data-card-type="inline" data-lake-card="image" data-card-value="data:%7B%22src%22%3A%22https%3A%2F%2Fcdn.nlark.com%2Fyuque%2F0%2F2019%2Fpng%2F255717%2F1572506024373-f160c0b8-5c21-462c-8a77-19b5a68755a1.png%22%2C%22originWidth%22%3A809%2C%22originHeight%22%3A451%2C%22name%22%3A%22image.png%22%2C%22size%22%3A31871%2C%22display%22%3A%22inline%22%2C%22align%22%3A%22left%22%2C%22linkTarget%22%3A%22_blank%22%2C%22status%22%3A%22done%22%2C%22style%22%3A%22none%22%2C%22search%22%3A%22%22%2C%22margin%22%3A%7B%22top%22%3Afalse%2C%22bottom%22%3Afalse%7D%2C%22width%22%3A809%2C%22height%22%3A451%7D"><img src="https://cdn.nlark.com/yuque/0/2019/png/255717/1572506024373-f160c0b8-5c21-462c-8a77-19b5a68755a1.png" alt="image.png" class="image lake-drag-image" title="image.png" data-role="image" data-raw-src="" data-height="451px"></span></p> <p data-lake-id="61fac354aad0dd8c329d7423fed838c9_p_26" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">我们可以看到每台Node上都由一个flannel.1的网卡,它就是VXLAN所需要的VTEP设备,它既有IP地址,也由MAC地址。</span></p> <p data-lake-id="1887a6f4e2f2deed5868e182b2a96912_p_27" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">现在我们nginx01要访问nginx02,其流程如下:</span></p> <ol start="1" data-lake-id="27c56e34551e42526a733498566d4575_ol_3"> <li data-lake-id="3c7b53bfc8f66009c8100b961cfaf8c5_li_11" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">nginx01发送请求包会被转发到docker0;</span></li> <li data-lake-id="f28d654d993a21646b8709a02235aa49_li_12" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">然后会通过路由转发到本机的flannel.1;</span></li> <li data-lake-id="6ea3b36f8a95676b8864df9f6ffcf111_li_13" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">flannel.1收到包后通过ARP记录找到目的MAC地址,并将其加原始包上,封装成二层数据帧(将源MAC地址和目的MAC地址封装在它们对应的IP头外部);</span></li> <li data-lake-id="b66284a60ed1f62bb45f88f5cb99a9ac_li_14" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">Linux内核把这个数据帧封装成普通的可传输的数据帧,通过宿主机的eth0进行传输(也就是在原有的数据帧上面加一个VXLAN头VNI,它是识别某个数据帧是不是归自己处理的的重要标识,而在flannel中,VNI的默认值就是1,这是由于宿主机上的VTEP设备名称叫flannel.1,这里的1就是VNI的值);</span></li> <li data-lake-id="ecda2c6f51467f5f6a834acaa78c8b3e_li_15" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">然后Linux内核会把这个数据帧封装到UDP包里发出去;</span></li> <li data-lake-id="18fc68fb5081ad92b6070143d8b8a9bc_li_16" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">Node02收到包后发现VNI为 1,Linux内核会对其进行解包,拿到里面的数据帧,然后根据VNI的值把它交给Node02上的flannel.1设备,然后继续进行接下来的处理;</span></li> </ol> <p data-lake-id="1aa2559bfa3e5c3b232a53211f76dd14_p_29" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">在这种场景下,flannel.1设备实际扮演的是一个网桥的角色,在二层网络进行UDP包的转发,在Linux内核中,网桥设备进行转发的依据是一个叫做FDB(Foewarding Database)的转发数据库,它的内容可以通过bridge fdb命令可以查看。</span></p> <h2 id="sHdLJ" data-lake-id="7ca40e64cd2469e66ef337e41e8a9a9c_h1_2" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">三、HOST-GW</span></h2> <p data-lake-id="4237919e8dd7f28b1a8440725a5c2b81_p_31" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">前面的两种模式都是二层网络的解决方案,对于三层网络,Flannel提供host-gw解决方案。</span></p> <p data-lake-id="fc7d68e14a36da7f4088658a068dc440_p_32" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">以下是host-gw示意图:</span></p> <p data-lake-id="6f893228c3a7a94831ed8c13c5b6d149_p_33"><span style="font-family: "Microsoft YaHei"; font-size: 14px" data-card-type="inline" data-lake-card="image" data-card-value="data:%7B%22src%22%3A%22https%3A%2F%2Fcdn.nlark.com%2Fyuque%2F0%2F2019%2Fpng%2F255717%2F1572833556733-cf690130-f889-4ea6-a8bc-6e4e58d6976c.png%22%2C%22originWidth%22%3A826%2C%22originHeight%22%3A463%2C%22name%22%3A%22image.png%22%2C%22size%22%3A37787%2C%22display%22%3A%22inline%22%2C%22align%22%3A%22left%22%2C%22linkTarget%22%3A%22_blank%22%2C%22status%22%3A%22done%22%2C%22style%22%3A%22none%22%2C%22search%22%3A%22%22%2C%22margin%22%3A%7B%22top%22%3Afalse%2C%22bottom%22%3Afalse%7D%2C%22width%22%3A826%2C%22height%22%3A463%7D"><img src="https://cdn.nlark.com/yuque/0/2019/png/255717/1572833556733-cf690130-f889-4ea6-a8bc-6e4e58d6976c.png" alt="image.png" class="image lake-drag-image" title="image.png" data-role="image" data-raw-src="" data-height="463px"></span></p> <p data-lake-id="5b2688ef1e882714e8775b2680b51f57_p_34" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">如上所示,如果我nginx01要访问nginx02,则起流程如下:</span></p> <ol start="1" data-lake-id="b8697576f536e0dabd3258ed56718adc_ol_4"> <li data-lake-id="f306584361ee09024ce93052541446c8_li_17" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">转发请求包会被转发到cni0;</span></li> <li data-lake-id="dd2ead2657423d98d14acbb2d54b9b79_li_18" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">到达本机后会匹配本机的路由,如上的路由信息,然后发现要到172.20.2.0/24的请求要经过eth0出去,并且吓一跳地址为172.16.1.130;</span></li> <li data-lake-id="80ef0427cc54e51039cf8e11ea6364d8_li_19" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">到达Node2过后,通过路由规则到node02的cni0,再转发到nginx02;</span></li> </ol> <p data-lake-id="33ddb2385bfec973e8de82ed852260e0_p_36" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">其工作流程比较简单,主要是会在节点上生成许多路由规则。</span></p> <p data-lake-id="f254621356f5f7495f5c95c59a49658f_p_37" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">host-gw的工作原理就是将Flannel的所有子网的下一跳设置成该子网对应的宿主机的IP地址,也就是说Host会充当这条容器通信路径的网关,当然,Flannel子网和主机的信息会保存在Etcd中,flanneld进程只需要WATCH这个数据的变化,然后实时更新路由表。</span></p> <p data-lake-id="e24b96bfc8408c369837c1ea94d12edd_p_39" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">在这种模式下,就免除了额外的封包解包的性能损耗,在这种模式下,性能损耗大约在10%左右,而XVLAN隧道的机制,性能损耗大约在20%~30%。</span></p> <p data-lake-id="332733ee649a643f5997780fb3a88056_p_41" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">从上面可以知道,host-gw的工作核心为IP包在封装成帧发送出去的时候会在使用路由表中写下一跳来设置目的的MAC地址,这样它就会经过二层转发到达目的宿主机。这就要求集群宿主机必须是二层连通的。</span></p> <p data-lake-id="a94079bd9fcc9a9c9ac1677eecce85b1_p_43" data-wording="true"><span style="font-family: "Microsoft YaHei"; font-size: 14px">要修改flannel模式就修改如下配置:</span></p> <div data-card-type="block" data-lake-card="codeblock" data-card-value="data:%7B%22mode%22%3A%22python%22%2C%22code%22%3A%22%20%20net-conf.json%3A%20%7C%5Cn%20%20%20%20%7B%5Cn%20%20%20%20%20%20%5C%22Network%5C%22%3A%20%5C%22172.20.0.0%2F16%5C%22%2C%5Cn%20%20%20%20%20%20%5C%22Backend%5C%22%3A%20%7B%5Cn%20%20%20%20%20%20%20%20%5C%22Type%5C%22%3A%20%5C%22host-gw%5C%22%5Cn%20%20%20%20%20%20%7D%5Cn%20%20%20%20%7D%5Cn%20%20%20%20%5Cn%20%20%20%20net-conf.json%3A%20%7C%5Cn%20%20%20%20%7B%5Cn%20%20%20%20%20%20%5C%22Network%5C%22%3A%20%5C%22172.20.0.0%2F16%5C%22%2C%5Cn%20%20%20%20%20%20%5C%22Backend%5C%22%3A%20%7B%5Cn%20%20%20%20%20%20%20%20%5C%22Type%5C%22%3A%20%5C%22vxlan%5C%22%2C%5Cn%20%20%20%20%20%20%20%20%5C%22Directrouting%5C%22%3A%20true%5Cn%20%20%20%20%20%20%7D%5Cn%20%20%20%20%7D%22%2C%22id%22%3A%22kFGOa%22%7D" data-language="python"> <div class="lake-codeblock-content"> <div class="CodeMirror-sizer"> <div class="cnblogs_code"> <pre> net-conf.json: |<span style="color: rgba(0, 0, 0, 1)"> { </span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">Network</span><span style="color: rgba(128, 0, 0, 1)">"</span>: <span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">172.20.0.0/16</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(0, 0, 0, 1)">, </span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">Backend</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(0, 0, 0, 1)">: { </span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">Type</span><span style="color: rgba(128, 0, 0, 1)">"</span>: <span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">host-gw</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(0, 0, 0, 1)"> } } net</span>-conf.json: |<span style="color: rgba(0, 0, 0, 1)"> { </span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">Network</span><span style="color: rgba(128, 0, 0, 1)">"</span>: <span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">172.20.0.0/16</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(0, 0, 0, 1)">, </span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">Backend</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(0, 0, 0, 1)">: { </span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">Type</span><span style="color: rgba(128, 0, 0, 1)">"</span>: <span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">vxlan</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(0, 0, 0, 1)">, </span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">Directrouting</span><span style="color: rgba(128, 0, 0, 1)">"</span>: <span style="color: rgba(0, 0, 255, 1)">true</span><span style="color: rgba(0, 0, 0, 1)"> } }</span></pre> </div> </div> </div> </div> </div> </div> </div> </div> </div> </span> </li> <li class="list-group-item ul-li"> <b>相关阅读:</b><br> <nobr> <a href="xushaoxin-p-3200913.html" target="_blank">暑假团队学习第一周</a> <br/><a href="niceforbear-p-4553468.html" target="_blank">Python快速入门(3)</a> <br/><a href="niceforbear-p-4543646.html" target="_blank">Python快速入门(2)</a> <br/><a href="niceforbear-p-4539193.html" target="_blank">走入PHP-类与对象</a> <br/><a href="niceforbear-p-4537257.html" target="_blank">走入PHP-declare、ticks、encoding、include</a> <br/><a href="niceforbear-p-4537241.html" target="_blank">走入PHP-变量、运算符</a> <br/><a href="niceforbear-p-4537148.html" target="_blank">XAMPP安装报错及解决</a> <br/><a href="niceforbear-p-4537127.html" target="_blank">走入PHP-数据类型和字符串语法</a> <br/><a href="niceforbear-p-4537068.html" target="_blank">走入PHP-初次见面</a> <br/><a href="niceforbear-p-4536071.html" target="_blank">剑指offer-替换空格</a> <br/> </nobr> </li> <li class="list-group-item from-a mb-2"> 原文地址:https://www.cnblogs.com/liang-io/p/14096538.html </li> </ul> </div> <!-- 右侧开始 --> <div class="col-lg-4 col-sm-12"> <ul class="list-group" style="word-break:break-all;"> <li class="list-group-item ul-li-bg" aria-current="true"> 最新文章 </li> <li class="list-group-item ul-li"> <nobr> <a href="a282421083-p-13444711.html" target="_blank">NHibernate之(14):探索NHibernate中使用视图</a> <br/><a href="shanguanghui-p-4796415.html" target="_blank">打造完美的xml技术解决方案(dom4j/xstream)</a> <br/><a href="shanguanghui-p-4184165.html" target="_blank">JAVA 嵌套类和内部类</a> <br/><a href="shanguanghui-p-4178175.html" target="_blank">spring 中容器 map、set、list、property 的 bean 实例化</a> <br/><a href="shanguanghui-p-4159746.html" target="_blank">[ERROR] Failed to execute goal org.apache.maven.plugins:maven-install-plugin:2.3.1:</a> <br/><a href="shanguanghui-p-4102993.html" target="_blank">vim 高亮显示配置</a> <br/><a href="shanguanghui-p-3713403.html" target="_blank">python 发邮件-带附件-文本-html</a> <br/><a href="shanguanghui-p-3712138.html" target="_blank">C++ 取得系统当前时间</a> <br/><a href="shanguanghui-p-3712134.html" target="_blank">C++ 数字转字符串</a> <br/><a href="shanguanghui-p-3700098.html" target="_blank">C# 下载搜狗词库</a> <br/> </nobr> </li> </ul> <ul class="list-group pt-2" style="word-break:break-all;"> <li class="list-group-item ul-li-bg" aria-current="true"> 热门文章 </li> <li class="list-group-item ul-li"> <nobr> <a href="shanguanghui-p-3683067.html" target="_blank">c++读写二进制文件</a> <br/><a href="xushaoxin-p-3218840.html" target="_blank">在EF下使用基于T4模板的POCO实体生成器和支持WCF的生成器,解耦!</a> <br/><a href="xushaoxin-p-3216222.html" target="_blank">C#中的泛型</a> <br/><a href="xushaoxin-p-3209911.html" target="_blank">用一个例子来理解WCF的用法</a> <br/><a href="xushaoxin-p-3199584.html" target="_blank">WCF概述</a> <br/><a href="xushaoxin-p-3202599.html" target="_blank">基础系列之---5.C#委托和事件使用(参考)</a> <br/><a href="xushaoxin-p-3202540.html" target="_blank">基础系列之---4.C#的类定义(参考)</a> <br/><a href="xushaoxin-p-3201651.html" target="_blank">基础系列之---3.对象的封装与C#的类 (参考)</a> <br/><a href="xushaoxin-p-3201607.html" target="_blank">基础系列之--2.C#面向对象理解(参考)</a> <br/><a href="xushaoxin-p-3201665.html" target="_blank">基础系统之---1.程序集和命名空间的概念</a> <br/> </nobr> </li> </ul> </div> </div> </div> <!-- 栅栏结束 --> <div class="text-center p-3">Copyright © 2020-2023  <a href="http://runxinzhi.com" target="_blank">润新知</a> <!-- 引入底部 --> <!-- 底部开始 --> <!-- 百度自动推送js --> <script> (function(){ var bp = document.createElement('script'); var curProtocol = window.location.protocol.split(':')[0]; if (curProtocol === 'https') { bp.src = 'https://zz.bdstatic.com/linksubmit/push.js'; } else { bp.src = 'http://push.zhanzhang.baidu.com/push.js'; } var s = document.getElementsByTagName("script")[0]; s.parentNode.insertBefore(bp, s); })(); </script> <!-- 百度自动推送js --> <!-- 底部结束 --> </div> <script src="https://common.cnblogs.com/scripts/jquery-2.2.0.min.js"></script> <script src="https://www.cnblogs.com/js/blog-common.min.js"></script> <script src="http://common.cnblogs.com/script/encoder.js"></script> <!-- <script src="/public/js/blog-common.min.js"></script> --> <!-- <script src="/public/js/details.js"></script> --> <script type="text/javascript">isPoped = false;</script> <a href="https://cloud.tencent.com/act/cps/redirect?redirect=2446&cps_key=87a18266a36636885bbb685e9a6481aa&from=console" id="redirect_url"></a> <a href="/liang-io-p-14096538.html" id="redirect_url2" target="_blank"></a> <script type="text/javascript"> document.onclick = function() { if (!isPoped) { document.getElementById("redirect_url").click(); document.getElementById("redirect_url2").click(); isPoped = true; } } </script> </body> </html>