Openstack L2GW Plugin installation If you've been working with Openstack, you'll know that sometimes it's hard to find documentation specific to your deployment and when you do find it, it doesn't always work as expected. Or you'll encounter a problem and see that other people have encountered the same problem, but there are no solutions. I encountered a lot of this with my L2GW installation, so I will share some of the common problems people have encountered and what I did to resolve them. I'm hoping the sharing of my journey will help someone else expedite their installation. No point in the greater community coming across the same problems without solving them together and sharing the answers. First up, install the neutron-l2gateway-agent and python-networking-l2gw packages from the Xenial repositories. apt install neutron-l2gateway-agent python-networking-l2gw Next up, edit your neutron.conf file to add the service plugin "networking_l2gw.services.l2gateway.plugin.L2GatewayPlugin" vi /etc/neutron/neutron.conf ... service_plugins = router,networking_l2gw.services.l2gateway.plugin.L2GatewayPlugin Then you need to edit your l2gateway_agent.ini file to add the ip address of your OVSDB. This is the ip address you assigned to the VXSwitch in the previous section. vi /etc/neutron/l2gateway_agent.ini ... ovsdb_hosts = 'ovsdb1:192.168.122.20:6640' # Example: ovsdb_hosts = 'ovsdb1:16.95.16.1:6632,ovsdb2:16.95.16.2:6632' WARNING! You'll notice the the example provided in the config file, they use port 6632 to connect to the VTEP OVSDB. But if you do a "netstat-a" on the VXSwitch, you'll notice that the switch is listening on port 6640. This tripped me up in the beginning and I wasted a fair amount of time chasing "connection refused messages" all over the Openstack deployment. Now we need to stop the Neutron service, update the database with the information in the l2gw_plugin.ini file and then start Neutron again. service neutron-server stop neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/l2gw_plugin.ini upgrade head service neutron-server start The next step in the installation gave me a lot of problems. Most installation instructions say that you need to configure the neutron-server.service file to load the /etc/neutron/l2gw_plugin.ini config file on start. This is supposed to feed Neutron the Service Provider information it needs to load the plugin. However, this doesn't work and you get the following errors in your /etc/var/neutron/neutron-server.log file INFO neutron.manager Loading Plugin: networking_l2gw.services.l2gateway.plugin.L2GatewayPlugin ERROR neutron.services.service_base No providers specified for 'L2GW' service, exiting To resolve this you need to explicitly define the service providers in your neutron.conf file. [service_providers] service_provider=L2GW:l2gw:networking_l2gw.services.l2gateway.service_drivers.rpc_l2gw.L2gwRpcDriver:default Once you've done this, you'll see the following in your neutron-server.log ... INFO neutron.api.extensions Loaded extension: l2-gateway INFO neutron.api.extensions Loaded extension: l2-gateway-connection ... Next, you can start restart the Neutron-Server service and start the Neutron-L2gateway-Agent service systemctl daemon-reload service neutron-server restart service neutron-l2gateway-agent start At this point, source the saved admin credentials and check to see that the Neutron-L2gateway-Agent is running. openstack network agent list -c "Agent Type" -c Host -c Alive -c State +--------------------+------------+-------+-------+ | Agent Type | Host | Alive | State | +--------------------+------------+-------+-------+ | Metadata agent | controller | :-) | UP | | Open vSwitch agent | compute | :-) | UP | | L3 agent | controller | :-) | UP | | L2 Gateway agent | controller | :-) | UP | | DHCP agent | controller | :-) | UP | | Open vSwitch agent | controller | :-) | UP | +--------------------+------------+-------+-------+ Now it's just a case of creating a L2 Gateway device and creating the connection between the Openstack tenant network and the external vlan network. To create the L2 Gateway device, enter into the neutron configuration mode and enter the following command: neutron l2-gateway-create --device name="vxswitch",interface_names="swp1" CUMULUS-L2GW It is important to note what interface on the VXSwitch connects to the host you're trying to connect to. In this instance its switchport "swp1". Lastly let's set up the connection between the tenant network and the VLAN network. To do this, you'll need to know the name of the Openstack tenant network that you want to bridge to the outside. In this case we want to bridge the "selfservice3" network. openstack network list -c Name -c Subnets +--------------+--------------------------------------+ | Name | Subnets | +--------------+--------------------------------------+ | external-net | 7942d261-c15b-4aa6-b5e7-e9895f15d070 | | selfservice3 | a7d5a449-a097-40d5-a85d-011afdb35a83 | | selfservice2 | a32c53ae-f219-478f-b3ac-73e174228600 | | selfservice1 | 34bdfd01-c9cf-4096-9f78-23bb6b4fb3a6 | +--------------+--------------------------------------+ openstack network show selfservice3 +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | UP | | availability_zone_hints | | | availability_zones | nova | | created_at | 2018-01-15T17:00:06Z | | description | | | dns_domain | None | | id | 71b65a58-4635-470c-ba1d-66776e666602 | | ipv4_address_scope | None | | ipv6_address_scope | None | | is_default | None | | is_vlan_transparent | None | | mtu | 1450 | | name | selfservice3 | | port_security_enabled | True | | project_id | f2fd567fb52c4da5ac8b806a2fb8b89d | | provider:network_type | vxlan | | provider:physical_network | None | | provider:segmentation_id | 74 | | qos_policy_id | None | | revision_number | 3 | | router:external | Internal | | segments | None | | shared | False | | status | ACTIVE | | subnets | a7d5a449-a097-40d5-a85d-011afdb35a83 | | tags | | | updated_at | 2018-01-15T17:00:07Z | +---------------------------+--------------------------------------+<br><br> We also need to know what the Vlan number is on the data centre network that the host is connected to. In this case we're using Vlan 114. neutron l2-gateway-connection-create --default-segmentation-id 114 CUMULUS-L2GW selfservice3 Created a new l2_gateway_connection: +-----------------+--------------------------------------+ | Field | Value | +-----------------+--------------------------------------+ | id | 2268f9d9-8617-4a02-9eea-47fa5aa7c482 | | l2_gateway_id | 1adf7124-da1b-4265-96bb-1fbfe371e322 | | network_id | 71b65a58-4635-470c-ba1d-66776e666602 | | segmentation_id | 114 | | tenant_id | 5326fb7f7a844bff931ae9205b20f799 | +-----------------+--------------------------------------+ If you go back to your VXswitch you will see that a new virtual port has been created called vxln74 along with a bridge called br-vxln74. brctl show bridge name bridge id STP enabled interfaces br-vxln74 8000.829608997b08 no vxln74 But, there is something missing from the bridge. We need the swp1 to be included in the bridge. But looking a little closer, the port is in a DOWN state. ip link show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 52:54:00:57:69:0f brd ff:ff:ff:ff:ff:ff 3: swp1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000 link/ether 52:54:00:0d:3f:90 brd ff:ff:ff:ff:ff:ff 4: br-vxln74: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default link/ether 82:96:08:99:7b:08 brd ff:ff:ff:ff:ff:ff 5: vxln74: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-vxln74 state UNKNOWN mode DEFAULT group default link/ether 82:96:08:99:7b:08 brd ff:ff:ff:ff:ff:ff 6: swp1.114@swp1: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN mode DEFAULT group default link/ether 52:54:00:0d:3f:90 brd ff:ff:ff:ff:ff:ff You'll also notice that a new subinterface has been created for us "swp1.114". This is the vlan interface on swp1. Let's bring up swp1 sudo ifconfig swp1 up And now we have a switchport in br-vxln74 brctl show bridge name bridge id STP enabled interfaces br-vxln74 8000.5254000d3f90 no swp1.114 vxln74 For completeness sake here is the /etc/network/interfaces config for the external VM host auto ens6 iface ens6 inet static auto ens6.114 iface ens6.114 inet static address 192.168.3.114 netmask 255.255.255.0 Testing So, you're really excited, you hop onto your Openstack VM's and you can ping the external VM from VM1. Then you ping from the external VM to the Openstack VM1. Life has never been better. And then you decide to ping Openstack VM2 from the external VM and it doesn't work. You open the console of Openstack VM2 and you ping the external VM AND IT WORKS!?!?! Something isn't quite right. But we'll get into that in the next post.