UE can't ping outide the NAT(internet for exemple)

[kouamdo]

Hi , i'm building every network node(open5gs) using kubernetes and docker.That is the architecture, and i want to know why i can't ping to internet :

i'm facing one issue , is that , the UE can't ping to outside the NAT , i have made some modification but nothing.

there , you can see that the PDU session was created :

thenre is the SMF

that are the logs when i trying to ping outide the NAT : ( ping -c1 google.com -I uesimtun0 )
eUPF :

NAT gateway :

So , let me know where i have mistaken.

[kouamdo]

by using this configuration inside the NAT gateway

i have this inside the eUPF :

and inside the NAT_GW i have this :

i think that the problem is inside the NAT_GW but i didn't find it

[pirog-spb]

Hi @kouamdo

According to your last screenshort, GTP packets are sent to NAT GW instead of gNB.

eUPF uses kernel routing table to forward packets. So ensure that you have a route to gNB in the linux routing table(in container).

Here eUPF used gw 172.16.0.1 to send packets towards gNB(10.1.229.71).

Seems like gw 172.16.0.1 sent packets to NAT GW instead of gNB. Check routing on the host as well.

[kouamdo]

they take this road because i have add this route : ip r add default via 172.16.0.13 dev eth1 inside the eUPF container
So is it necessary to add route for gNB ?
Or maybe , that default route should be removed and replaced by another one🤔

[kouamdo]

that is the last one , after making some change :

and inside the eUPF :

[pirog-spb]

@kouamdo Did you get bpf_fib_lookup result 7(BPF_FIB_LKUP_RET_NO_NEIGH) every time packet was processed?

[kouamdo]

@kouamdo Did you get bpf_fib_lookup result 7(BPF_FIB_LKUP_RET_NO_NEIGH) every time packet was processed?

Yes , sure...
Nothing is working fine until now

[pirog-spb]

Usually, first packet gets BPF_FIB_LKUP_RET_NO_NEIGH because ARP table is empty. But subsequent packets are routed well.

Try to ping IP-address manually(to fill arp table) form eupf container and check routing after that.

[pirog-spb]

We have to localize the problem first. My point - the problem is in routing settings.

The simplest routing config in eupf would be as follows:

iptables -A FORWARD -j ACCEPT
echo "1200 n6if" >> /etc/iproute2/rt_tables
ip rule add from 10.45.0.0/16 table n6if
ip route add default via 172.16.0.13 dev eth1 table n6if

So, ip rule for N3->N6 packets and generic default route(not shown here) for N3->N6 packets

[pirog-spb]

Now the routing error is BPF_FIB_LKUP_RET_NOT_FWDED(4)

enum {
	BPF_FIB_LKUP_RET_SUCCESS,      /* lookup successful */
	BPF_FIB_LKUP_RET_BLACKHOLE,    /* dest is blackholed; can be dropped */
	BPF_FIB_LKUP_RET_UNREACHABLE,  /* dest is unreachable; can be dropped */
	BPF_FIB_LKUP_RET_PROHIBIT,     /* dest not allowed; can be dropped */
	BPF_FIB_LKUP_RET_NOT_FWDED,    /* packet is not forwarded */
	BPF_FIB_LKUP_RET_FWD_DISABLED, /* fwding is not enabled on ingress */
	BPF_FIB_LKUP_RET_UNSUPP_LWT,   /* fwd requires encapsulation */
	BPF_FIB_LKUP_RET_NO_NEIGH,     /* no neighbor entry for nh */
	BPF_FIB_LKUP_RET_FRAG_NEEDED,  /* fragmentation required to fwd */
};

There is still some issue in routing settings outside of eUPF scope of responsibility. eUPF during routing just calls kernel helper to get route for the packet.

BTW, gNB address has been changed. Is it ok?

In order to debug routing you can use ip utility:

ip r get 10.240.233.71 from 172.18.0.2

Or even:

ip r get 10.240.233.71 from 172.18.0.2 iif eth1

If it's acceptable we may have a short conf call to find out what's going wrong.

[kouamdo]

it is fine now , the issue was that i have change the default route and change the gateway , .... so i could live it

but let me show you some informations that i have :

there is the docker compose file :

version: '2.4'

services:
  eupf:
    image: local/eupf:latest
    entrypoint:
      - /bin/sh
      - -c
      - |
        mkdir -p /etc/iproute2/;
        echo "1000 n6if" >> /etc/iproute2/rt_tables;
        ip rule add from 10.33.0.0/16 table n6if;
        ip route add default via 172.16.0.13 dev eth2 table n6if &&
        sh /app/bin/entrypoint.sh --config /app/bin/eupf_config.yml
    privileged: true
    environment:
      - GIN_MODE=release

    volumes:
      - /sys/fs/bpf:/sys/fs/bpf
      - /sys/kernel/debug:/sys/kernel/debug:ro
      - ./eupf_config.yml:/app/bin/eupf_config.yml
    ulimits:
      memlock: -1
    cap_add:
      - NET_ADMIN
      - SYS_ADMIN
    ports:
      - 2152:2152/udp
      - 8805:8805/udp
      - 8081:8081
      - 9091:9091
    restart: unless-stopped
    networks:
      n3:
        ipv4_address: 172.18.0.12
      n4:
        ipv4_address: 172.19.0.12
      n6:
        ipv4_address: 172.16.0.12
    sysctls:
      - net.ipv4.conf.all.forwarding=1

  nat-dn:
    image: ubuntu:focal
    privileged: true
    restart: unless-stopped
    networks:
      n6:
        ipv4_address: 172.16.0.13
    command:
      - /bin/sh
      - -c
      - |
        apt update && apt install -y iproute2 tcpdump iptables
        sysctl -w net.ipv6.conf.all.disable_ipv6=1
        echo 1 > /proc/sys/net/ipv4/ip_forward
        iptables -t nat -A POSTROUTING -s 10.33.0.0/16 -j MASQUERADE
        ip ro add 10.33.0.0/16 via 172.16.0.12 dev eth0
        echo "done"
        tail -f /dev/null

networks:
  n3:
    external: true
  n4:
    external: true
  n6:
    external: true

that's about route

that's inside the DN :

[kouamdo]

i have tried with native , bit there is no XDP program attached to driver interface. Also , the ping is not working.

and i'm using this version of kernel 5.16.0-051600-generic

why i don't have XDP programattached to interface inside ? Or let me know if i should know something