Deploying Deep Learning

Welcome to our training guide for inference and deep vision runtime library for NVIDIA DIGITS and Jetson Xavier/TX1/TX2.

This repo uses NVIDIA TensorRT for efficiently deploying neural networks onto the embedded platform, improving performance and power efficiency using graph optimizations, kernel fusion, and half-precision FP16 on the Jetson.

Vision primitives, such as imageNet for image recognition, detectNet for object localization, and segNet for segmentation, inherit from the shared tensorNet object. Examples are provided for streaming from live camera feed and processing images from disk. See the Deep Vision API Reference Specification for accompanying documentation.

>   See Image Segmentation models and training guide with aerial drone dataset.
>   Try Object Detection training guide using DIGITS & MS-COCO training dataset.
>   Use Image Recognition training guide with DIGITS & ImageNet ILSVRC12 dataset.
>   Check out our next GitHub tutorial, Deep Reinforcement Learning in Robotics.
>   Jetson AGX Xavier Developer Kit and JetPack 4.1.1 DP is now available and supported in the repo.

Table of Contents

• DIGITS Workflow
• System Setup
• Building from Source on Jetson
• Classifying Images with ImageNet
  • Using the Console Program on Jetson
  • Running the Live Camera Recognition Demo
  • Re-training the Network with DIGITS
  • Downloading Image Recognition Dataset
  • Customizing the Object Classes
  • Importing Classification Dataset into DIGITS
  • Creating Image Classification Model with DIGITS
  • Testing Classification Model in DIGITS
  • Downloading Model Snapshot to Jetson
  • Loading Custom Models on Jetson
• Locating Object Coordinates using DetectNet
  • Detection Data Formatting in DIGITS
  • Downloading the Detection Dataset
  • Importing the Detection Dataset into DIGITS
  • Creating DetectNet Model with DIGITS
  • Testing DetectNet Model Inference in DIGITS
  • Downloading the Model Snapshot to Jetson
  • DetectNet Patches for TensorRT
  • Processing Images from the Command Line on Jetson
  • Multi-class Object Detection Models
  • Running the Live Camera Detection Demo on Jetson
• Image Segmentation with SegNet
  • Downloading Aerial Drone Dataset
  • Importing the Aerial Dataset into DIGITS
  • Generating Pretrained FCN-Alexnet
  • Training FCN-Alexnet with DIGITS
  • Testing Inference Model in DIGITS
  • FCN-Alexnet Patches for TensorRT
  • Running Segmentation Models on Jetson

Recommended System Requirements

Training GPU: Maxwell, Pascal, or Volta-based GPU (ideally with at least 6GB video memory)
                        optionally, AWS P2/P3 instance or Microsoft Azure N-series
                        Ubuntu 14.04 x86_64 or Ubuntu 16.04 x86_64.

Deployment:   Jetson Xavier Developer Kit with JetPack 4.0 or newer (Ubuntu 18.04 aarch64).
                        Jetson TX2 Developer Kit with JetPack 3.0 or newer (Ubuntu 16.04 aarch64).
                        Jetson TX1 Developer Kit with JetPack 2.3 or newer (Ubuntu 16.04 aarch64).

note: this branch is verified against the following BSP versions for Jetson AGX Xavier and Jetson TX1/TX2:
             > Jetson AGX Xavier - JetPack 4.1.1 DP / L4T R31.1 aarch64 (Ubuntu 18.04 LTS) inc. TensorRT 5.0 GA
             > Jetson AGX Xavier - JetPack 4.1 DP EA / L4T R31.0.2 aarch64 (Ubuntu 18.04 LTS) inc. TensorRT 5.0 RC
             > Jetson AGX Xavier - JetPack 4.0 DP EA / L4T R31.0.1 aarch64 (Ubuntu 18.04 LTS) inc. TensorRT 5.0 RC
             > Jetson TX2 - JetPack 3.3 / L4T R28.2.1 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 4.0
             > Jetson TX1 - JetPack 3.3 / L4T R28.2 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 4.0
             > Jetson TX2 - JetPack 3.2 / L4T R28.2 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 3.0
             > Jetson TX2 - JetPack 3.1 / L4T R28.1 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 3.0 RC
             > Jetson TX1 - JetPack 3.1 / L4T R28.1 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 3.0 RC
             > Jetson TX2 - JetPack 3.1 / L4T R28.1 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 2.1
             > Jetson TX1 - JetPack 3.1 / L4T R28.1 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 2.1
             > Jetson TX2 - JetPack 3.0 / L4T R27.1 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 1.0
             > Jetson TX1 - JetPack 2.3 / L4T R24.2 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 1.0
             > Jetson TX1 - JetPack 2.3.1 / L4T R24.2.1 aarch64 (Ubuntu 16.04 LTS)

Note that TensorRT samples from the repo are intended for deployment onboard Jetson, however when cuDNN and TensorRT have been installed on the host side, the TensorRT samples in the repo can be compiled for PC.

Extra Resources

In this area, links and resources for deep learning developers are listed:

• Appendix
  • ros_deep_learning - TensorRT inference ROS nodes
  • NVIDIA AI IoT - NVIDIA Jetson GitHub repositories
  • Jetson eLinux Wiki - Jetson eLinux Wiki

Legacy Links

Since the documentation has been re-organized, below are links mapping the previous content to the new locations.

       (click on the arrow above to hide this section)

DIGITS Workflow

See DIGITS Workflow

System Setup

See DIGITS Setup

Running JetPack on the Host

See JetPack Setup

Installing Ubuntu on the Host

See DIGITS Setup

Setting up host training PC with NGC container

See DIGITS Setup

Installing the NVIDIA driver

See DIGITS Setup

Installing Docker

See DIGITS Setup

NGC Sign-up

See DIGITS Setup

Setting up data and job directories

See DIGITS Setup

Starting DIGITS container

See DIGITS Setup

Natively setting up DIGITS on the Host

See DIGITS Native Setup

Installing NVIDIA Driver on the Host

See DIGITS Native Setup

Installing cuDNN on the Host

See DIGITS Native Setup

Installing NVcaffe on the Host

See DIGITS Native Setup

Installing DIGITS on the Host

See DIGITS Native Setup

Starting the DIGITS Server

See DIGITS Native Setup

Building from Source on Jetson

See Building the Repo from Source

Cloning the Repo

See Building the Repo from Source

Configuring with CMake

See Building the Repo from Source

Compiling the Project

See Building the Repo from Source

Digging Into the Code

See Building the Repo from Source

Classifying Images with ImageNet

See Classifying Images with ImageNet

Using the Console Program on Jetson

See Classifying Images with ImageNet

Running the Live Camera Recognition Demo

See Running the Live Camera Recognition Demo

Re-training the Network with DIGITS

See Re-Training the Recognition Network

Downloading Image Recognition Dataset

See Re-Training the Recognition Network

Customizing the Object Classes

See Re-Training the Recognition Network

Importing Classification Dataset into DIGITS

See Re-Training the Recognition Network

Creating Image Classification Model with DIGITS

See Re-Training the Recognition Network

Testing Classification Model in DIGITS

See Re-Training the Recognition Network

Downloading Model Snapshot to Jetson

See Downloading Model Snapshots to Jetson

Loading Custom Models on Jetson

See Loading Custom Models on Jetson

Locating Object Coordinates using DetectNet

See Locating Object Coordinates using DetectNet

Detection Data Formatting in DIGITS

See Locating Object Coordinates using DetectNet

Downloading the Detection Dataset

See Locating Object Coordinates using DetectNet

Importing the Detection Dataset into DIGITS

See Locating Object Coordinates using DetectNet

Creating DetectNet Model with DIGITS

See Locating Object Coordinates using DetectNet

Selecting DetectNet Batch Size

See Locating Object Coordinates using DetectNet

Specifying the DetectNet Prototxt

See Locating Object Coordinates using DetectNet

Training the Model with Pretrained Googlenet

See Locating Object Coordinates using DetectNet

Testing DetectNet Model Inference in DIGITS

See Locating Object Coordinates using DetectNet

Downloading the Model Snapshot to Jetson

See Downloading the Detection Model to Jetson

DetectNet Patches for TensorRT

See Downloading the Detection Model to Jetson

Processing Images from the Command Line on Jetson

See Detecting Objects from the Command Line

Launching With a Pretrained Model

See Detecting Objects from the Command Line

Pretrained DetectNet Models Available

See Detecting Objects from the Command Line

Running Other MS-COCO Models on Jetson

See Detecting Objects from the Command Line

Running Pedestrian Models on Jetson

See Detecting Objects from the Command Line

Multi-class Object Detection Models

See Detecting Objects from the Command Line

Running the Live Camera Detection Demo on Jetson

See Running the Live Camera Detection Demo

Image Segmentation with SegNet

See Semantic Segmentation with SegNet

Downloading Aerial Drone Dataset

See Semantic Segmentation with SegNet

Importing the Aerial Dataset into DIGITS

See Semantic Segmentation with SegNet

Generating Pretrained FCN-Alexnet

See Generating Pretrained FCN-Alexnet

Training FCN-Alexnet with DIGITS

See Training FCN-Alexnet with DIGITS

Testing Inference Model in DIGITS

See Training FCN-Alexnet with DIGITS

FCN-Alexnet Patches for TensorRT

See FCN-Alexnet Patches for TensorRT

Running Segmentation Models on Jetson

See Running Segmentation Models on Jetson