Adding 3 nvidia convnets models (pytorch#250)

* Adding 3 nvidia convnets models * Fixed sanity_check.py * Fixed tags * Added model training information to the summary * fix broken link * Fix missing featured_image_2 * fixing missing parenthesis * resolve missing featured_image_2 in efficientnet * Fixed rxn title and python cells Co-authored-by: Vincent Moens <[email protected]>
Lelontar391 · Nov 16, 2021 · 33f8731 · 33f8731
1 parent a069374
commit 33f8731
Show file tree

Hide file tree

Showing 5 changed files with 344 additions and 0 deletions.
diff --git a/images/ResNeXtArch.png b/images/ResNeXtArch.png
diff --git a/images/classification.jpg b/images/classification.jpg
diff --git a/nvidia_deeplearningexamples_efficientnet.md b/nvidia_deeplearningexamples_efficientnet.md
@@ -0,0 +1,120 @@
+---
+layout: hub_detail
+background-class: hub-background
+body-class: hub
+title: EfficientNet
+summary: EfficientNets are a family of image classification models, which achieve state-of-the-art accuracy, being an order-of-magnitude smaller and faster. Trained with mixed precision using Tensor Cores.
+category: researchers
+image: nvidia_logo.png
+author: NVIDIA
+tags: [vision]
+github-link: https://github.com/NVIDIA/DeepLearningExamples/tree/master/PyTorch/Classification/ConvNets/efficientnet
+github-id: NVIDIA/DeepLearningExamples
+featured_image_1: classification.jpg
+featured_image_2: none
+accelerator: cuda
+order: 10
+---
+
+
+### Model Description
+
+EfficientNet is an image classification model family. It was first described in [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://arxiv.org/abs/1905.11946). This notebook allows you to load and test the EfficientNet-B0, EfficientNet-B4, EfficientNet-WideSE-B0 and, EfficientNet-WideSE-B4 models.
+
+EfficientNet-WideSE models use Squeeze-and-Excitation layers wider than original EfficientNet models, the width of SE module is proportional to the width of Depthwise Separable Convolutions instead of block width.
+
+WideSE models are slightly more accurate than original models.
+
+This model is trained with mixed precision using Tensor Cores on Volta and the NVIDIA Ampere GPU architectures. Therefore, researchers can get results over 2x faster than training without Tensor Cores, while experiencing the benefits of mixed precision training. This model is tested against each NGC monthly container release to ensure consistent accuracy and performance over time.
+
+We use [NHWC data layout](https://pytorch.org/tutorials/intermediate/memory_format_tutorial.html) when training using Mixed Precision.
+
+### Example
+
+In the example below we will use the pretrained ***EfficientNet*** model to perform inference on image and present the result.
+
+To run the example you need some extra python packages installed. These are needed for preprocessing images and visualization.
+```python
+!pip install validators
+```
+
+```python
+import torch
+from PIL import Image
+import torchvision.transforms as transforms
+import numpy as np
+import json
+import requests
+
+device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
+print(f'Using {device} for inference')
+```
+
+Load the model pretrained on IMAGENET dataset.
+
+By setting *type* parameter you can choose among the following models:
+
+| Type | Description |
+| :----- | :----- |
+| `efficientnet-b0` | baseline EfficientNet |
+| `efficientnet-b4` | scaled EfficientNet|
+| `efficientnet-widese-b0` | model with Squeeze-and-Excitation layers wider than baseline EfficientNet model |
+| `efficientnet-widese-b4` | model with Squeeze-and-Excitation layers wider than scaled EfficientNet model |
+
+There are also quantized version of the models, but they require nvidia container. See [quantized models](https://github.com/NVIDIA/DeepLearningExamples/tree/master/PyTorch/Classification/ConvNets/efficientnet#quantization)
+```python
+efficientnet = torch.hub.load('NVIDIA/DeepLearningExamples:torchhub', 'nvidia_efficientnet', type='efficientnet-widese-b0')
+utils = torch.hub.load('NVIDIA/DeepLearningExamples:torchhub', 'nvidia_convnets_processing_utils')
+
+efficientnet.eval().to(device)
+
+```
+
+Prepare sample input data.
+```python
+uris = [
+    'http://images.cocodataset.org/test-stuff2017/000000024309.jpg',
+    'http://images.cocodataset.org/test-stuff2017/000000028117.jpg',
+    'http://images.cocodataset.org/test-stuff2017/000000006149.jpg',
+    'http://images.cocodataset.org/test-stuff2017/000000004954.jpg',
+]
+
+batch = torch.cat(
+    [utils.prepare_input_from_uri(uri) for uri in uris]
+).to(device)
+```
+
+Run inference. Use `pick_n_best(predictions=output, n=topN)` helper function to pick N most probable hypotheses according to the model.
+```python
+with torch.no_grad():
+    output = torch.nn.functional.softmax(efficientnet(batch), dim=1)
+
+results = utils.pick_n_best(predictions=output, n=5)
+```
+
+Display the result.
+```python
+for uri, result in zip(uris, results):
+    img = Image.open(requests.get(uri, stream=True).raw)
+    img.thumbnail((256,256), Image.ANTIALIAS)
+    img.show()
+    print(result)
+```
+
+### Details
+For detailed information on model input and output, training recipies, inference and performance visit:
+[github](https://github.com/NVIDIA/DeepLearningExamples/tree/master/PyTorch/Classification/ConvNets/efficientnet)
+and/or [NGC](https://ngc.nvidia.com/catalog/resources/nvidia:efficientnet_for_pytorch)
+
+### References
+
+ - [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://arxiv.org/abs/1905.11946)
+ - [model on NGC](https://ngc.nvidia.com/catalog/resources/nvidia:efficientnet_for_pytorch)
+ - [model on github](https://github.com/NVIDIA/DeepLearningExamples/tree/master/PyTorch/Classification/ConvNets/efficientnet)
+ - [pretrained model on NGC (efficientnet-b0)](https://ngc.nvidia.com/catalog/models/nvidia:efficientnet_b0_pyt_amp)
+ - [pretrained model on NGC (efficientnet-b4)](https://ngc.nvidia.com/catalog/models/nvidia:efficientnet_b4_pyt_amp)
+ - [pretrained model on NGC (efficientnet-widese-b0)](https://ngc.nvidia.com/catalog/models/nvidia:efficientnet_widese_b0_pyt_amp)
+ - [pretrained model on NGC (efficientnet-widese-b4)](https://ngc.nvidia.com/catalog/models/nvidia:efficientnet_widese_b4_pyt_amp)
+ - [pretrained, quantized model on NGC (efficientnet-widese-b0)](https://ngc.nvidia.com/catalog/models/nvidia:efficientnet_widese_b0_pyt_amp)
+ - [pretrained, quantized model on NGC (efficientnet-widese-b4)](https://ngc.nvidia.com/catalog/models/nvidia:efficientnet_widese_b4_pyt_amp)
+
diff --git a/nvidia_deeplearningexamples_resneXt.md b/nvidia_deeplearningexamples_resneXt.md
@@ -0,0 +1,115 @@
+---
+layout: hub_detail
+background-class: hub-background
+body-class: hub
+title: ResNeXt101
+summary: ResNet with bottleneck 3x3 Convolutions substituted by 3x3 Grouped Convolutions, trained with mixed precision using Tensor Cores.
+category: researchers
+image: nvidia_logo.png
+author: NVIDIA
+tags: [vision]
+github-link: https://github.com/NVIDIA/DeepLearningExamples/tree/master/PyTorch/Classification/ConvNets/resnext101-32x4d
+github-id: NVIDIA/DeepLearningExamples
+featured_image_1: ResNeXtArch.png
+featured_image_2: classification.jpg
+accelerator: cuda
+order: 10
+---
+
+
+### Model Description
+
+The ***ResNeXt101-32x4d*** is a model introduced in the [Aggregated Residual Transformations for Deep Neural Networks](https://arxiv.org/pdf/1611.05431.pdf) paper.
+
+It is based on regular ResNet model, substituting 3x3 convolutions inside the bottleneck block for 3x3 grouped convolutions.
+
+This model is trained with mixed precision using Tensor Cores on Volta, Turing, and the NVIDIA Ampere GPU architectures. Therefore, researchers can get results 3x faster than training without Tensor Cores, while experiencing the benefits of mixed precision training. This model is tested against each NGC monthly container release to ensure consistent accuracy and performance over time.
+
+We use [NHWC data layout](https://pytorch.org/tutorials/intermediate/memory_format_tutorial.html) when training using Mixed Precision.
+
+Note that the ResNeXt101-32x4d model can be deployed for inference on the [NVIDIA Triton Inference Server](https://github.com/NVIDIA/trtis-inference-server) using TorchScript, ONNX Runtime or TensorRT as an execution backend. For details check [NGC](https://ngc.nvidia.com/catalog/resources/nvidia:resnext_for_triton_from_pytorch)
+
+#### Model architecture
+
+![ResNextArch](ResNeXtArch.png)
+
+_Image source: [Aggregated Residual Transformations for Deep Neural Networks](https://arxiv.org/pdf/1611.05431.pdf)_
+
+Image shows difference between ResNet bottleneck block and ResNeXt bottleneck block.
+
+ResNeXt101-32x4d model's cardinality equals to 32 and bottleneck width equals to 4.
+### Example
+
+In the example below we will use the pretrained ***ResNeXt101-32x4d*** model to perform inference on images and present the result.
+
+To run the example you need some extra python packages installed. These are needed for preprocessing images and visualization.
+```python
+!pip install validators
+```
+
+```python
+import torch
+from PIL import Image
+import torchvision.transforms as transforms
+import numpy as np
+import json
+import requests
+
+device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
+print(f'Using {device} for inference')
+```
+
+Load the model pretrained on IMAGENET dataset.
+```python
+resneXt = torch.hub.load('NVIDIA/DeepLearningExamples:torchhub', 'nvidia_resneXt')
+utils = torch.hub.load('NVIDIA/DeepLearningExamples:torchhub', 'nvidia_convnets_processing_utils')
+
+resneXt.eval().to(device)
+```
+
+Prepare sample input data.
+```python
+uris = [
+    'http://images.cocodataset.org/test-stuff2017/000000024309.jpg',
+    'http://images.cocodataset.org/test-stuff2017/000000028117.jpg',
+    'http://images.cocodataset.org/test-stuff2017/000000006149.jpg',
+    'http://images.cocodataset.org/test-stuff2017/000000004954.jpg',
+]
+
+
+batch = torch.cat(
+    [utils.prepare_input_from_uri(uri) for uri in uris]
+).to(device)
+```
+
+Run inference. Use `pick_n_best(predictions=output, n=topN)` helepr function to pick N most probably hypothesis according to the model.
+```python
+with torch.no_grad():
+    output = torch.nn.functional.softmax(resneXt(batch), dim=1)
+
+results = utils.pick_n_best(predictions=output, n=5)
+```
+
+Display the result.
+```python
+for uri, result in zip(uris, results):
+    img = Image.open(requests.get(uri, stream=True).raw)
+    img.thumbnail((256,256), Image.ANTIALIAS)
+    img.show()
+    print(result)
+
+```
+
+### Details
+For detailed information on model input and output, training recipies, inference and performance visit:
+[github](https://github.com/NVIDIA/DeepLearningExamples/tree/master/PyTorch/Classification/ConvNets/resnext101-32x4d)
+and/or [NGC](https://ngc.nvidia.com/catalog/resources/nvidia:resnext_for_pytorch)
+
+
+### References
+
+ - [Aggregated Residual Transformations for Deep Neural Networks](https://arxiv.org/pdf/1611.05431.pdf)
+ - [model on github](https://github.com/NVIDIA/DeepLearningExamples/tree/master/PyTorch/Classification/ConvNets/resnext101-32x4d)
+ - [model on NGC](https://ngc.nvidia.com/catalog/resources/nvidia:resnext_for_pytorch)
+ - [pretrained model on NGC](https://ngc.nvidia.com/catalog/models/nvidia:resnext101_32x4d_pyt_amp)
+
diff --git a/nvidia_deeplearningexamples_resnet50.md b/nvidia_deeplearningexamples_resnet50.md
@@ -0,0 +1,109 @@
+---
+layout: hub_detail
+background-class: hub-background
+body-class: hub
+title: ResNet50
+summary: ResNet50 model trained with mixed precision using Tensor Cores.
+category: researchers
+image: nvidia_logo.png
+author: NVIDIA
+tags: [vision]
+github-link: https://github.com/NVIDIA/DeepLearningExamples/tree/master/PyTorch/Classification/ConvNets/resnet50v1.5
+github-id: NVIDIA/DeepLearningExamples
+featured_image_1: classification.jpg
+featured_image_2: no-image
+accelerator: cuda
+order: 10
+---
+
+
+### Model Description
+
+The ***ResNet50 v1.5*** model is a modified version of the [original ResNet50 v1 model](https://arxiv.org/abs/1512.03385).
+
+The difference between v1 and v1.5 is that, in the bottleneck blocks which requires
+downsampling, v1 has stride = 2 in the first 1x1 convolution, whereas v1.5 has stride = 2 in the 3x3 convolution.
+
+This difference makes ResNet50 v1.5 slightly more accurate (\~0.5% top1) than v1, but comes with a smallperformance drawback (\~5% imgs/sec).
+
+The model is initialized as described in [Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification](https://arxiv.org/pdf/1502.01852.pdf)
+
+This model is trained with mixed precision using Tensor Cores on Volta, Turing, and the NVIDIA Ampere GPU architectures. Therefore, researchers can get results over 2x faster than training without Tensor Cores, while experiencing the benefits of mixed precision training. This model is tested against each NGC monthly container release to ensure consistent accuracy and performance over time.
+
+Note that the ResNet50 v1.5 model can be deployed for inference on the [NVIDIA Triton Inference Server](https://github.com/NVIDIA/trtis-inference-server) using TorchScript, ONNX Runtime or TensorRT as an execution backend. For details check [NGC](https://ngc.nvidia.com/catalog/resources/nvidia:resnet_for_triton_from_pytorch)
+
+
+### Example
+
+In the example below we will use the pretrained ***ResNet50 v1.5*** model to perform inference on ***image*** and present the result.
+
+To run the example you need some extra python packages installed. These are needed for preprocessing images and visualization.
+```python
+!pip install validators
+```
+
+```python
+import torch
+from PIL import Image
+import torchvision.transforms as transforms
+import numpy as np
+import json
+import requests
+
+device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
+print(f'Using {device} for inference')
+```
+
+Load the model pretrained on IMAGENET dataset.
+```python
+resnet50 = torch.hub.load('NVIDIA/DeepLearningExamples:torchhub', 'nvidia_resnet50')
+utils = torch.hub.load('NVIDIA/DeepLearningExamples:torchhub', 'nvidia_convnets_processing_utils')
+
+resnet50.eval().to(device)
+```
+
+Prepare sample input data.
+```python
+uris = [
+    'http://images.cocodataset.org/test-stuff2017/000000024309.jpg',
+    'http://images.cocodataset.org/test-stuff2017/000000028117.jpg',
+    'http://images.cocodataset.org/test-stuff2017/000000006149.jpg',
+    'http://images.cocodataset.org/test-stuff2017/000000004954.jpg',
+]
+
+batch = torch.cat(
+    [utils.prepare_input_from_uri(uri) for uri in uris]
+).to(device)
+```
+
+Run inference. Use `pick_n_best(predictions=output, n=topN)` helepr function to pick N most probably hypothesis according to the model.
+```python
+with torch.no_grad():
+    output = torch.nn.functional.softmax(resnet50(batch), dim=1)
+
+results = utils.pick_n_best(predictions=output, n=5)
+```
+
+Display the result.
+```python
+for uri, result in zip(uris, results):
+    img = Image.open(requests.get(uri, stream=True).raw)
+    img.thumbnail((256,256), Image.ANTIALIAS)
+    img.show()
+    print(result)
+
+```
+
+### Details
+For detailed information on model input and output, training recipies, inference and performance visit:
+[github](https://github.com/NVIDIA/DeepLearningExamples/tree/master/PyTorch/Classification/ConvNets/resnet50v1.5)
+and/or [NGC](https://ngc.nvidia.com/catalog/resources/nvidia:resnet_50_v1_5_for_pytorch)
+
+### References
+
+ - [Original ResNet50 v1 paper](https://arxiv.org/abs/1512.03385)
+ - [Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification](https://arxiv.org/pdf/1502.01852.pdf)
+ - [model on github](https://github.com/NVIDIA/DeepLearningExamples/tree/master/PyTorch/Classification/ConvNets/resnet50v1.5)
+ - [model on NGC](https://ngc.nvidia.com/catalog/resources/nvidia:resnet_50_v1_5_for_pytorch)
+ - [pretrained model on NGC](https://ngc.nvidia.com/catalog/models/nvidia:resnet50_pyt_amp)
+