Demikernel

Demikernel is a library operating system (LibOS) architecture designed for use with kernel-bypass I/O devices. This architecture offers a uniform system call API across kernel-bypass technologies (e.g., RDMA, DPDK) and OS functionality (e.g., a user-level networking stack for DPDK).

To read more about the motivation behind the Demikernel, check out this blog post.

To get details about the system, read our paper in SOSP '21.

To read more about Demikernel check out https://aka.ms/demikernel.

Building

Follow these instructions to build Demikernel on a fresh Ubuntu 20.04 system.

1. Clone This Repository

export WORKDIR=$HOME                                                  # Change this to whatever you want.
cd $WORKDIR                                                           # Switch to working directory.
git clone --recursive https://github.com/demikernel/demikernel.git    # Recursive clone.
cd $WORKDIR/demikernel                                                # Switch to repository's source tree.

2. Install Prerequisites (Only Once)

sudo -H scripts/setup/debian.sh                                   # Install third party libraries.
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh    # Get Rust toolchain.

3. Build DPDK Libraries (For Catnip and Only Once)

./scripts/setup/dpdk.sh

4. Build Demikernel with Default Parameters

make

5. Build Demikernel with Custom Parameters (Optional)

make LIBOS=[catnap|catnip|catpowder|catcollar]    # Build using a specific LibOS.
make DRIVER=[mlx4|mlx5]                           # Build using a specific driver.
make LD_LIBRARY_PATH=/path/to/libs                # Override path to shared libraries. Applicable to Catnap and Catcollar.
make PKG_CONFIG_PATH=/path/to/pkgconfig           # Override path to config files. Applicable to Catnap and Catcollar.

6. Install Artifacts (Optional)

make install                                     # Copies build artifacts to your $HOME directory.
make install INSTALL_PREFIX=/path/to/location    # Copies build artifacts to a specific location.

Running

Follow these instructions to run examples that are shipped in the source tree.

1. Setup Configuration File (Only Once)

• Copy the template from scripts/config/default.yaml to $HOME/config.yaml.
• Open the file in $HOME/config.yaml for editing and do the following:
  • Change XX.XX.XX.XX to match the IPv4 address of your server host.
  • Change YY.YY.YY.YY to match the IPv4 address of your client host.
  • Change PPPP to the port number that you will expose in the server host.
  • Change ZZ.ZZ.ZZ.ZZ to match the IPv4 address that in the local host.
  • Change ff:ff:ff:ff:ff:ff to match the MAC address in the local host.
  • Change abcde to match the name of the interface in the local host.
  • Change the arp_table according to your setup.
  • If using DPDK, change WW:WW.W to match the PCIe address of your NIC.
• Save the file.

2. Enable Huge Pages (For Catnip at Every System Reboot)

sudo -E ./scripts/setup/hugepages.sh

3. Run UDP Push-Pop Demo

For Catnap and Catcollar, you don't need to run with super-user privileges.

# Server-Side
PEER=server TEST=udp_push_pop sudo -E make LIBOS=catnip test-system

# Client-Side
PEER=client TEST=udp_push_pop sudo -E make LIBOS=catnip test-system

4. Run UDP Ping-Pong Demo

For Catnap and Catcollar, you don't need to run with super-user privileges.

# Server-Side
PEER=server TEST=udp_ping_pong sudo -E make LIBOS=catnip test-system

# Client-Side
PEER=client TEST=udp_ping_pong sudo -E make LIBOS=catnip test-system

Documentation

• Legacy system call API documentation doc/syscalls.md
• Instructions for running Demikernel on CloudLab doc/cloudlab.md

1. Build API Documentation (Optional)

cargo doc --no-deps    # Build API Documentation
cargo doc --open       # Open API Documentation

Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Contributing

See CONTRIBUTING for details regarding how to contribute to this project.

Usage Statement

This project is a prototype. As such, we provide no guarantees that it will work and you are assuming any risks with using the code. We welcome comments and feedback. Please send any questions or comments to one of the following maintainers of the project:

• Irene Zhang - [email protected]
• Pedro Henrique Penna - [email protected]

By sending feedback, you are consenting that it may be used in the further development of this project.