Skip to content

Commit

Permalink
Config support for OVS-DPDK
Browse files Browse the repository at this point in the history
  • Loading branch information
ipatrykx committed Jun 3, 2020
1 parent 7324fc6 commit 8121afd
Show file tree
Hide file tree
Showing 4 changed files with 426 additions and 6 deletions.
6 changes: 6 additions & 0 deletions dist/images/install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -491,6 +491,8 @@ spec:
name: host-log-ovs
- mountPath: /var/log/ovn
name: host-log-ovn
- mountPath: /opt/ovs-config
name: host-config-ovs
- mountPath: /dev/hugepages
name: hugepage
readinessProbe:
Expand Down Expand Up @@ -542,6 +544,10 @@ spec:
- name: host-log-ovn
hostPath:
path: /var/log/ovn
- name: host-config-ovs
hostPath:
path: /opt/ovs-config
type: DirectoryOrCreate
- name: hugepage
emptyDir:
medium: HugePages
Expand Down
36 changes: 30 additions & 6 deletions dist/images/start-ovs-dpdk.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,36 @@ function quit {
}
trap quit EXIT

# Start ovsdb and vswitchd
ovs-ctl --no-ovs-vswitchd start
ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-socket-mem="1024"
ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true
ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-hugepage-dir=/dev/hugepages
ovs-ctl --no-ovsdb-server start
CONFIG_FILE=/opt/ovs-config/config.cfg

# Check if config file exists, create default one if not
if ! test -f "$CONFIG_FILE"; then
printf %s\\n {dpdk-socket-mem=\"1024\",dpdk-init=true,dpdk-hugepage-dir=/dev/hugepages} > $CONFIG_FILE
fi

# Start ovsdb
ovs-ctl restart --no-ovs-vswitchd --system-id=random

# Restrict the number of pthreads ovs-vswitchd creates to reduce the
# amount of RSS it uses on hosts with many cores
# https://bugzilla.redhat.com/show_bug.cgi?id=1571379
# https://bugzilla.redhat.com/show_bug.cgi?id=1572797
if [[ `nproc` -gt 12 ]]; then
ovs-vsctl --no-wait set Open_vSwitch . other_config:n-revalidator-threads=4
ovs-vsctl --no-wait set Open_vSwitch . other_config:n-handler-threads=10
fi

# Read the config and setup OVS
while IFS= read -r config_line
do
if [[ $config_line ]] && [[ $config_line != \#* ]] ; then
ovs-vsctl --no-wait set Open_vSwitch . other_config:$config_line
fi
done < "$CONFIG_FILE"

# Start vswitchd
ovs-ctl restart --no-ovsdb-server --system-id=random
ovs-ctl --protocol=udp --dport=6081 enable-protocol

# Start ovn-controller
ovn-ctl restart_controller
Expand Down
27 changes: 27 additions & 0 deletions docs/dpdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,33 @@ Hugepagesize: 1048576 kB
```


## Optional configuration

Open vSwitch is highly configurable using `other_config` options as described in [Open vSwitch Manual](http://www.openvswitch.org/support/dist-docs/ovs-vswitchd.conf.db.5.txt).
All of those configs can be configured using simple config file `/opt/ovs-config/config.cfg`. This file is being mounted in `ovs-ovn` pod. It contains list of `other_config` options. Each option should be placed in new line.

Example:

```
dpdk-socket-mem="1024,1024"
dpdk-init=true
pmd-cpu-mask=0x4
dpdk-lcore-mask=0x2
dpdk-hugepage-dir=/dev/hugepages
```

This example config will enable DPDK support with 1024MB of hugepages for both NUMA node 0 and NUMA node 1, PMD CPU mask 0x4, lcore mask 0x2 and hugepages in /dev/hugepages.

If file will not exist upon OVS initialization, the default configuration file will be created with values:

```
dpdk-socket-mem="1024"
dpdk-init=true
dpdk-hugepage-dir=/dev/hugepages
```
>**Note:** Please remember, that if you would like to initialize Open vSwitch with more socket memory than 1024MB, you will have to reserve this memory for `ovs-ovn` pod by editing the value `hugepages-1G` of `ovs-ovn` pod in `install.sh` script. For example, to initialize Open vSwitch using `dpdk-socket-mem="1024,1024"` the minimal value will be `hugepages-1G: 2Gi`.

## To Install

1. Download the installation script:
Expand Down
Loading

0 comments on commit 8121afd

Please sign in to comment.