UPF dpdk-mode with virtio-net don't process downlink traffic

[v0rdych]

Hi, I'm trying to run UPF inside KVM machine via DPDK mode, where network interfaces are emulated by KVM and are connected to Linux bridge on the host machine.
UPF stars normally, DPDK seems to be working:

docker logs 1f109e4db845

I0502 07:30:45.988786 1 main.cc:64] Launching BESS daemon in process mode...
I0502 07:30:45.988965 1 main.cc:77] bessd v0.4.0-304-g7460e68d-dirty
I0502 07:30:45.991011 1 bessd.cc:458] Loading plugin (attempt 1): /bin/modules/sequential_update.so
I0502 07:30:45.994578 1 memory.cc:128] /sys/devices/system/node/possible not available. Assuming a single-node system...
I0502 07:30:45.994601 1 dpdk.cc:169] Initializing DPDK EAL with options: ["bessd", "--main-lcore", "127", "--lcore", "127@2-3", "--no-shconf", "--legacy-mem", "--socket-mem", "1024", "--huge-unlink"]
EAL: Detected 10 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Detected static linkage of DPDK
EAL: Selected IOVA mode 'PA'
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: Probe PCI driver: net_virtio (1af4:1041) device: 0000:01:00.0 (socket 0)
EAL: Probe PCI driver: net_virtio (1af4:1041) device: 0000:06:00.0 (socket 0)
EAL: using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_virtio (1af4:1041) device: 0000:07:00.0 (socket 0)
EAL: No legacy callbacks, legacy socket not created
Segment 0-0: IOVA:0xbc0000000, len:1073741824, virt:0x140000000, socket_id:0, hugepage_sz:1073741824, nchannel:0, nrank:0 fd:10
I0502 07:30:51.657445 1 packet_pool.cc:49] Creating DpdkPacketPool for 262144 packets on node 0
I0502 07:30:51.657510 1 packet_pool.cc:70] PacketPool0 requests for 262144 packets
I0502 07:30:51.729945 1 packet_pool.cc:157] PacketPool0 has been created with 262144 packets
I0502 07:30:51.730376 1 pmd.cc:70] 2 DPDK PMD ports have been recognized:
I0502 07:30:51.730449 1 pmd.cc:94] DPDK port_id 0 (net_virtio) RXQ 1 TXQ 1 de:b9:f3:67:1c:95 00000000:06:00.00 1af4:1041 numa_node 0
I0502 07:30:51.730516 1 pmd.cc:94] DPDK port_id 1 (net_virtio) RXQ 1 TXQ 1 7a:13:f5:fb:58:c8 00000000:07:00.00 1af4:1041 numa_node 0
I0502 07:30:51.730624 1 vport.cc:320] vport: BESS kernel module is not loaded. Loading...
sh: 1: insmod: not found
W0502 07:30:51.732473 1 vport.cc:332] Cannot load kernel module /bin/kmod/bess.ko
I0502 07:30:51.732537 1 bessctl.cc:1928] Server listening on 0.0.0.0:10514
I0502 07:32:46.362602 43 bessctl.cc:489] *** All workers have been paused ***
I0502 07:32:46.875159 60 worker.cc:321] Worker 0(0x7f2ff7ffd400) is running on core 2 (socket 0)

Uplink works fine, it is decasulated from GTP-U on access interface, goes to Core interface, but answers on Core interface are dropped by PDRLookup module:

When session is established on UPF I see following errors in Bess-pfcpiface log

ERRO[2023-05-02T07:33:53Z]/pfcpiface/pfcpiface/bess.go:757 github.com/omec-project/upf-epc/pfcpiface.(*bess).processPDR() pdrLookup method failed with resp: error:{code:28 errmsg:"failed to add a new wildcard pattern"}, err:

And in Bess-container's log:

I0502 07:33:53.473553 429 dpdk.cc:74] HASH: memory allocation failed
I0502 07:33:53.476033 431 dpdk.cc:74] HASH: memory allocation failed

As far as I could get through the code it seems to me that some kind of initialization of hash table for rules fails on virtio-net device. But it is not clear for me should it work in theory or not?

[github-actions]

This issue has been stale for 30 days and will be closed in 5 days. Comment to keep it open.

[github-actions]

This issue has been stale for 30 days and will be closed in 5 days. Comment to keep it open.

[github-actions]

This issue has been stale for 30 days and will be closed in 5 days. Comment to keep it open.

[amarsri28]

you might have inserted rules for uplink but not for downlink. pls check.

[v0rdych]

you might have inserted rules for uplink but not for downlink. pls check.

Could you please clarify how to check content of PDR lookup table?
BTW, it seems yes, these rules are not installed but they are not installed because of some inconsistency of how DPDK lib is used in BESS and how virtio tap driver is implemented.

[github-actions]

This issue has been stale for 30 days and will be closed in 5 days. Comment to keep it open.

[github-actions]

This issue has been stale for 30 days and will be closed in 5 days. Comment to keep it open.

[github-actions]

This issue has been stale for 30 days and will be closed in 5 days. Comment to keep it open.

[github-actions]

This issue has been stale for 30 days and will be closed in 5 days. Comment to keep it open.

[github-actions]

This issue has been stale for 30 days and will be closed in 5 days. Comment to keep it open.

[github-actions]

This issue has been stale for 30 days and will be closed in 5 days. Comment to keep it open.