无法通过virtio在qemu进程中使用testpmd向虚拟机发送pkt

人气:160 发布:2023-01-03 标签: virtualization qemu dpdk

问题描述

我正在尝试测试vhost-user/virtio-net。我使用testpmd将pkt(在仅txonly模式下)发送到QEMU VM。但Testpmd显示所有Pkt均有下降。以下是我的环境:

DPDK version: 19.08
(HOST) Hugepagesize=1GB Hugepages=16
# testpmd cmd
testpmd -l 0-3 -n 4 --socket-mem 1024 --vdev 'net_vhost0,iface=/tmp/sock0,queues=1' -- -i
# qemu cmd
qemu-system-x86_64 /opt/vm/centos/vm.img 
        -cpu qemu64,+ssse3,+sse4.1,+sse4.2 
        --enable-kvm 
        --nographic -vnc :0 
        -smp 4 
        -m 4096 -mem-path /dev/hugepages,share=on -mem-prealloc 
        -chardev socket,id=chr0,path=/tmp/sock0 
        -netdev vhost-user,id=net0,chardev=chr0,queues=1,vhostforce 
        -device virtio-net-pci,netdev=net0,ioeventfd=on,mac=52:54:00:00:00:14 
        -netdev tap,id=tapnet0,ifname=tap1,script=no,downscript=no 
        -device e1000,netdev=tapnet0

已建立链接并初始化了虚拟队列:

VHOST_CONFIG: new vhost user connection is 31
VHOST_CONFIG: new device, handle is 0
VHOST_CONFIG: read message VHOST_USER_GET_FEATURES
VHOST_CONFIG: read message VHOST_USER_GET_PROTOCOL_FEATURES
VHOST_CONFIG: read message VHOST_USER_SET_PROTOCOL_FEATURES
VHOST_CONFIG: negotiated Vhost-user protocol features: 0x7
VHOST_CONFIG: read message VHOST_USER_GET_QUEUE_NUM
VHOST_CONFIG: read message VHOST_USER_SET_OWNER
VHOST_CONFIG: read message VHOST_USER_GET_FEATURES
VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL
VHOST_CONFIG: vring call idx:0 file:32
VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL
VHOST_CONFIG: vring call idx:1 file:33
VHOST_CONFIG: read message VHOST_USER_SET_VRING_ENABLE
VHOST_CONFIG: set queue enable: 1 to qp idx: 0

Port 0: queue state event
VHOST_CONFIG: read message VHOST_USER_SET_VRING_ENABLE
VHOST_CONFIG: set queue enable: 1 to qp idx: 1

Port 0: queue state event
VHOST_CONFIG: read message VHOST_USER_SET_FEATURES
VHOST_CONFIG: negotiated Virtio features: 0x7820ffc3
VHOST_CONFIG: read message VHOST_USER_SET_MEM_TABLE
VHOST_CONFIG: guest memory region 0, size: 0x40000000
     guest physical addr: 0x100000000
     guest virtual  addr: 0x7f9f7fe00000
     host  virtual  addr: 0x7f83d8000000
     mmap addr : 0x7f8318000000
     mmap size : 0x100000000
     mmap align: 0x1000
     mmap off  : 0xc0000000
VHOST_CONFIG: guest memory region 1, size: 0xa0000
     guest physical addr: 0x0
     guest virtual  addr: 0x7f9ebfe00000
     host  virtual  addr: 0x7f8421788000
     mmap addr : 0x7f8421788000
     mmap size : 0xa0000
     mmap align: 0x1000
     mmap off  : 0x0
VHOST_CONFIG: guest memory region 2, size: 0xbff40000
     guest physical addr: 0xc0000
     guest virtual  addr: 0x7f9ebfec0000
     host  virtual  addr: 0x7f82580c0000
     mmap addr : 0x7f8258000000
     mmap size : 0xc0000000
     mmap align: 0x1000
     mmap off  : 0xc0000
VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM
VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE
VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR
VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK
VHOST_CONFIG: vring kick idx:0 file:37
VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL
VHOST_CONFIG: vring call idx:0 file:38
VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM
VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE
VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR
VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK
VHOST_CONFIG: vring kick idx:1 file:32
VHOST_CONFIG: virtio is now ready for processing.

Port 0: link state change event
VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL
VHOST_CONFIG: vring call idx:1 file:39

在VM中设置NIC Promisc:

# ifconfig eth0 promisc up

testpmd显示链路状态为打开:

testpmd> show port info all

********************* Infos for port 0  *********************
MAC address: 56:48:4F:53:54:00
Device name: net_vhost0
Driver name: net_vhost
Devargs: iface=/tmp/sock0,queues=1
Connect to socket: 0
memory allocation on the socket: 0
Link status: up
Link speed: 10000 Mbps
Link duplex: full-duplex
MTU: 1500
Promiscuous mode: disabled
Allmulticast mode: disabled
Maximum number of MAC addresses: 1
Maximum number of MAC addresses of hash filtering: 0
VLAN offload: 
  strip off 
  filter off 
  qinq(extend) off 
No RSS offload flow type is supported.
Minimum size of RX buffer: 0
Maximum configurable length of RX packet: 4294967295
Current number of RX queues: 1
Max possible RX queues: 1
Max possible number of RXDs per queue: 65535
Min possible number of RXDs per queue: 0
RXDs number alignment: 1
Current number of TX queues: 1
Max possible TX queues: 1
Max possible number of TXDs per queue: 65535
Min possible number of TXDs per queue: 0
TXDs number alignment: 1
Max segment number per packet: 65535
Max segment number per MTU/TSO: 65535

testpmd启动发送包:

testpmd> set fwd txonly
Set txonly packet forwarding mode
testpmd> start

停止测试pmd:

Waiting for lcores to finish...

  ---------------------- Forward statistics for port 0  ----------------------
  RX-packets: 0              RX-dropped: 0             RX-total: 0
  TX-packets: 0              TX-dropped: 17056768      TX-total: 17056768
  ----------------------------------------------------------------------------

  +++++++++++++++ Accumulated forward statistics for all ports+++++++++++++++
  RX-packets: 0              RX-dropped: 0             RX-total: 0
  TX-packets: 0              TX-dropped: 17056768      TX-total: 17056768
  ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Done.

已丢弃所有包。

我错过了什么吗?

推荐答案

看起来有DPDK或NUMA支持的页面问题。同样适用于DPDK版本19.11 LTS和20.11 LTS。

DPDK应用: rm /tmp/sock0; sudo ./build/l2fwd --legacy-mem -l 1-2 --no-pci --vdev=net_vhost0,iface=/tmp/sock0 --vdev=net_tap0 -m 1024 -- -p 3 -T 1 --no-mac-updating

QEMU: taskset -c 4-9 qemu-system-x86_64 -cpu host -enable-kvm -m 1024 -smp 4,sockets=1,cores=4,threads=1 -object memory-backend-file,id=mem,size=1024M,mem-path=/mnt/huge,share=on -numa node,memdev=mem,nodeid=0 -mem-prealloc -name test -no-reboot -vnc none -nographic -net user,hostfwd=tcp::10023-:22 -net nic -chardev socket,id=charnet0,path=/tmp/sock0 -netdev type=vhost-user,chardev=charnet0,queues=1,id=hostnet0 -device virtio-net-pci,mq=on,vectors=18,netdev=hostnet0,id=net0,mac=fa:16:3e:52:30:73 -hda [disk name]

启动虚拟机后,您可以通过ssh port 10023登录。

19