一、编译dpdk lib
1、配置编译工具链
make config T=x86_64-native-linuxapp-gcc
2、编译
cd build
make
编译错误一:/root/dpdk-18.11/lib/librte_eal/linuxapp/eal/eal_memory.c:29:18: fatal error: numa.h: No such file or directory
原因:缺少库
解决方法:yum install numactl-devel*x86_64 -y
编译错误二:make: *** /lib/modules/3.10.0-957.1.3.el7.x86_64/build: No such file or directory. Stop.
原因:系统没有安装内核开发包
解决方法:(1)yum install kernel-devel -y (2)ln -s /usr/src/kernels/3.10.0-957.5.1.el7.x86_64 /usr/src/kernels/3.10.0-957.1.3.el7.x86_64
3、结果
编译出来的.a文件在目录/root/dpdk-18.11/build/lib下
二、dpdk用例
1、例1:hello world
(1)编译
cd /root/dpdk-18.11/examples/helloworld
export RTE_SDK=/root/dpdk-18.11
export RTE_TARGET=x86_64-native-linuxapp-gcc
export RTE_TARGET=build
make
会在当前目录生成build目录,里面有编译生成的可执行文件。
(2)运行helloworld
[root@localhost helloworld]# ./build/helloworld
EAL: Detected 4 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: No free hugepages reported in hugepages-2048kB
EAL: No free hugepages reported in hugepages-2048kB
EAL: FATAL: Cannot get hugepage information.
EAL: Cannot get hugepage information.
PANIC in main():
Cannot init EAL
5: [./build/helloworld() [0x4804df]]
4: [/lib64/libc.so.6(__libc_start_main+0xf5) [0x7fe22c67c3d5]]
3: [./build/helloworld() [0x4774b8]]
2: [./build/helloworld(__rte_panic+0xb8) [0x4609fb]]
1: [./build/helloworld(rte_dump_stack+0x1a) [0x5526ba]]
Aborted (core dumped)
运行环境配有配置,需要配置一下运行环境。
source usertools/dpdk-setup.sh,分别设置18,20,22
然后就可以正常运行了。
[root@localhost dpdk-18.11]# ./examples/helloworld/build/helloworld -l 0-1 -n 4
EAL: Detected 4 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Probing VFIO support...
hello from core 0
hello from core 1
2、例子2:Basic Forwarding/Skeleton Application
(1)编译完运行./build/basicfwd -l 1 -n 4
[root@localhost skeleton]# ./build/basicfwd -l 1 -n 4
EAL: Detected 4 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Probing VFIO support...
EAL: Error - exiting with code: 1
Cause: Error: number of ports must be even
原因:dpdk没有绑定网卡。
看下服务器的网卡信息,只有一块网卡。
[root@localhost skeleton]# ifconfig
enp3s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.19.8.93 netmask 255.255.255.0 broadcast 172.19.8.255
inet6 fe80::5246:5dff:fe5b:49be prefixlen 64 scopeid 0x20<link>
ether 50:46:5d:5b:49:be txqueuelen 1000 (Ethernet)
RX packets 765935 bytes 286280011 (273.0 MiB)
RX errors 0 dropped 3 overruns 0 frame 0
TX packets 128062 bytes 13559483 (12.9 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 22 bytes 1748 (1.7 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 22 bytes 1748 (1.7 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
下面我们着手创建两块虚拟网卡,查看dpdk支持的网卡列表,发现也支持tap虚拟网卡。
https://core.dpdk.org/supported/
支持虚拟网卡需要内核版本在4.2以上,可以通过以下命令查询是否支持虚拟tap:
zcat /proc/config.gz | ( grep 'CLS_FLOWER=' || echo 'not supported' ) |
tee -a /dev/stderr | grep -q '=m' &&
lsmod | ( grep cls_flower || echo 'try modprobe cls_flower' )
由于我们的系统内核不能升级,只能用virtualbox创建虚机时多加两块网卡。创建的虚机有三块网卡,下面把其中的两块绑定dpdk
(2)网卡绑定dpdk
[root@localhost dpdk-18.11]# ./usertools/dpdk-devbind.py --bind=uio_pci_generic enp0s8
Error - no supported modules(DPDK driver) are loaded
原因:uio驱动没有加载。回头看下,source usertools/dpdk-setup.sh,设置18时,报错了:## ERROR: Target does not have the DPDK UIO Kernel Module.To fix, please try to rebuild target.
解决方法:
[root@localhost dpdk-18.11]# modprobe uio
[root@localhost dpdk-18.11]# cd build/kmod/
[root@localhost kmod]# insmod igb_uio.ko
[root@localhost kmod]# lsmod | grep uio
再次运次
[root@localhost dpdk-18.11]# ./usertools/dpdk-devbind.py --bind=uio_pci_generic enp0s8
Routing table indicates that interface 0000:00:08.0 is active. Not modifying
原因:接口up无法绑定,先把期望绑定的接口down掉,再绑定。仍然报错。
[root@localhost dpdk-18.11]# ifconfig enp0s8 down
[root@localhost dpdk-18.11]# ./usertools/dpdk-devbind.py --bind=uio_pci_generic enp0s8
Error: bind failed for 0000:00:08.0 - Cannot open /sys/bus/pci/drivers/uio_pci_generic/bind
Error: unbind failed for 0000:00:08.0 - Cannot open /sys/bus/pci/drivers//unbind
[root@localhost dpdk-18.11]#
加载驱动 uio_pci_generic,继续绑定,仍然报错。
[root@localhost dpdk-18.11]# modprobe uio_pci_generic
[root@localhost dpdk-18.11]# lsmod | grep uio_pci_generic
uio_pci_generic 12588 0
uio 19338 2 uio_pci_generic,igb_uio
[root@localhost dpdk-18.11]# ./usertools/dpdk-devbind.py --bind=uio_pci_generic enp0s8
Unknown device: enp0s8. Please specify device in "bus:slot.func" format
查看一下网卡信息,发现enp0s8网卡不见了。查看一下状态,
用id好绑定:
[root@localhost dpdk-18.11]# ./usertools/dpdk-devbind.py --bind=uio_pci_generic 00:08.0
成功。
同样绑定第二块网卡。最终状态如下。
执行程序:
由于虚机和办公网络是通的,一下把办公网络搞挂了。抓包看全是ARP报文。
(3)下面对照例子代码,分析为什么用这个IP发包。
答案:其实这个例子本身没有发生三层报文,只是简单的转发了网卡上收到的ARP广播报文,这个报文并不是自身协议栈产生的。