Skip to content

Commit

Permalink
Add unittest on model zoo, and mobilenetv2_1.0 result (dmlc#82)
Browse files Browse the repository at this point in the history 
* add imagenet scripts to model zoo

* more resnet results

* add resnet v1

* fix table

* fix table

* fix table

* add download

* add new resnet18/34 models and logs

* correct numbers

* add resnet50 v1 logs

* add resnet101 logs

* improve modelzoo test

* add mobilenet result

* add reference for mobilenetv2
  • Loading branch information
@hetong007
hetong007 authored May 2, 2018
1 parent 3c70a24 commit bec7943
Show file tree
Hide file tree
Showing 2 changed files with 44 additions and 25 deletions.
55 changes: 30 additions & 25 deletions docs/model_zoo/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -57,29 +57,31 @@ Besides the listed, we provide more models trained on ImageNet in the upstream

Training commands work with this script: :download:`Download train_imagenet.py<../../scripts/classification/imagenet/train_imagenet.py>`

+-------------------+--------+--------+---------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------+
| Model | Top-1 | Top-5 | Training Command | Training Log |
+===================+========+========+=================================================================================================================================+===============================================================================================================================+
| ResNet18_v1 [1]_ | 0.7039 | 0.8959 | `shell script <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet18_v1.sh>`_ | `log <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet18_v1.log>`_ |
+-------------------+--------+--------+---------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------+
| ResNet34_v1 [1]_ | 0.7411 | 0.9184 | `shell script <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet34_v1.sh>`_ | `log <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet34_v1.log>`_ |
+-------------------+--------+--------+---------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------+
| ResNet50_v1 [1]_ | 0.7540 | 0.9266 | `shell script <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet50_v1.sh>`_ | `log <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet50_v1.log>`_ |
+-------------------+--------+--------+---------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------+
| ResNet101_v1 [1]_ | 0.7693 | 0.9334 | `shell script <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet101_v1.sh>`_ | `log <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet101_v1.log>`_ |
+-------------------+--------+--------+---------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------+
| ResNet152_v1 [2]_ | 0.7727 | 0.9353 | `shell script <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet152_v1.sh>`_ | |
+-------------------+--------+--------+---------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------+
| ResNet18_v2 [3]_ | 0.7037 | 0.8962 | `shell script <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet18_v2.sh>`_ | `log <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet18_v2.log>`_ |
+-------------------+--------+--------+---------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------+
| ResNet34_v2 [2]_ | 0.7389 | 0.9178 | `shell script <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet34_v2.sh>`_ | `log <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet34_v2.log>`_ |
+-------------------+--------+--------+---------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------+
| ResNet50_v2 [2]_ | 0.7622 | 0.9297 | `shell script <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet50_v2.sh>`_ | `log <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet50_v2.log>`_ |
+-------------------+--------+--------+---------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------+
| ResNet101_v2 [2]_ | 0.7747 | 0.9375 | `shell script <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet101_v2.sh>`_ | `log <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet101_v2.log>`_ |
+-------------------+--------+--------+---------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------+
| ResNet152_v2 [2]_ | 0.7833 | 0.9409 | `shell script <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet152_v2.sh>`_ | |
+-------------------+--------+--------+---------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------+
+-----------------------+--------+--------+------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------+
| Model | Top-1 | Top-5 | Training Command | Training Log |
+=======================+========+========+====================================================================================================================================+===============================================================================================================================+
| ResNet18_v1 [1]_ | 0.7039 | 0.8959 | `shell script <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet18_v1.sh>`_ | `log <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet18_v1.log>`_ |
+-----------------------+--------+--------+------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------+
| ResNet34_v1 [1]_ | 0.7411 | 0.9184 | `shell script <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet34_v1.sh>`_ | `log <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet34_v1.log>`_ |
+-----------------------+--------+--------+------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------+
| ResNet50_v1 [1]_ | 0.7540 | 0.9266 | `shell script <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet50_v1.sh>`_ | `log <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet50_v1.log>`_ |
+-----------------------+--------+--------+------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------+
| ResNet101_v1 [1]_ | 0.7693 | 0.9334 | `shell script <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet101_v1.sh>`_ | `log <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet101_v1.log>`_ |
+-----------------------+--------+--------+------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------+
| ResNet152_v1 [2]_ | 0.7727 | 0.9353 | `shell script <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet152_v1.sh>`_ | |
+-----------------------+--------+--------+------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------+
| ResNet18_v2 [3]_ | 0.7037 | 0.8962 | `shell script <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet18_v2.sh>`_ | `log <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet18_v2.log>`_ |
+-----------------------+--------+--------+------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------+
| ResNet34_v2 [2]_ | 0.7389 | 0.9178 | `shell script <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet34_v2.sh>`_ | `log <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet34_v2.log>`_ |
+-----------------------+--------+--------+------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------+
| ResNet50_v2 [2]_ | 0.7622 | 0.9297 | `shell script <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet50_v2.sh>`_ | `log <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet50_v2.log>`_ |
+-----------------------+--------+--------+------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------+
| ResNet101_v2 [2]_ | 0.7747 | 0.9375 | `shell script <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet101_v2.sh>`_ | `log <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet101_v2.log>`_ |
+-----------------------+--------+--------+------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------+
| ResNet152_v2 [2]_ | 0.7833 | 0.9409 | `shell script <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/resnet152_v2.sh>`_ | |
+-----------------------+--------+--------+------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------+
| MobileNetV2_1.0 [7]_ | 0.7159 | 0.9047 | `shell script <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/mobilenetv2_1.0.sh>`_ | `log <https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/logs/classification/imagenet/mobilenetv2_1.0.log>`_ |
+-----------------------+--------+--------+------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------+

**CIFAR10**

Expand Down Expand Up @@ -220,5 +222,8 @@ Table of pre-trained models for semantic segmentation and their performance.
Christian Szegedy, Scott Reed, Cheng-Yang Fu, Alexander C. Berg.
SSD: Single Shot MultiBox Detector. ECCV 2016.
.. [6] Long, Jonathan, Evan Shelhamer, and Trevor Darrell. \
"Fully convolutional networks for semantic segmentation." \
Proceedings of the IEEE conference on computer vision and pattern recognition. 2015.
"Fully convolutional networks for semantic segmentation." \
Proceedings of the IEEE conference on computer vision and pattern recognition. 2015.
.. [7] Sandler, Mark, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. \
"Inverted Residuals and Linear Bottlenecks: Mobile Networks for Classification, Detection and Segmentation." \
arXiv preprint arXiv:1801.04381 (2018).
14 changes: 14 additions & 0 deletions tests/unittests/test_model_zoo.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,20 @@

import gluoncv as gcv

def test_classification_models():
x = mx.random.uniform(shape=(2, 3, 32, 32))
cifar_models = [
'cifar_resnet20_v1', 'cifar_resnet56_v1', 'cifar_resnet110_v1',
'cifar_resnet20_v2', 'cifar_resnet56_v2', 'cifar_resnet110_v2',
'cifar_wideresnet16_10', 'cifar_wideresnet28_10', 'cifar_wideresnet40_8',
]
for model in cifar_models:
net = gcv.model_zoo.get_model(model)
with warnings.catch_warnings():
warnings.simplefilter("ignore")
net.initialize()
net(x)

def test_ssd_models():
x = mx.random.uniform(shape=(2, 3, 512, 768)) # allow non-squre and larger inputs
models = ['ssd_300_vgg16_atrous_voc', 'ssd_512_vgg16_atrous_voc', 'ssd_512_resnet50_v1_voc']
Expand Down

0 comments on commit bec7943

Please sign in to comment.