Image Classification Tutorial

Table of Contents

1. Updates
2. Features
3. Dependencies
4. Data
5. Baseline Results
6. Pretraining Results
7. FixRes Results
8. References

1. Updates 📰

• [2025.02] Support Reliability Metric: AURC ( 👍 AURC = 👍 classification accuracy + 👍 reliability)
• [2025.02] Add Deit-B and ConvNext-B Results
• [2024.02] Add FixRes training strategy
• [2024.01] Initial release with ResNet support

2. Features 🚥

• Supporting ConvNext, Deit; 🔥 For ConvNext, CIFAR10: 98.8%; CIFAR100: 92.4%; CUB: 91.22% ; CARS 93.32% 🔥
• Supporting Reliability Metric: AURC, see here for details
• 95.62% top-1 acc on CIFAR10, 79.11% top-1 acc on CIFAR100 with resnet18
• Support CIFAR10, CIFAR100, CUB and CARS
• Support standard retrained weight on Inet1K in supervised and self-supervised fashion (MOCOV2)
• Fixing the train-test resolution discrepancy
• Tensorboard Visualization, EMA, ONLY simple data aug

3. Dependencies 📚

The model can be trained on a single GPU with more than 12 GB of memory.

• Install PyTorch adapted to your CUDA version via Conda:

  conda install pytorch torchvision pytorch-cuda=11.8 -c pytorch -c nvidia

• Install TensorBoard:

  conda install matplotlib tensorboard

4. Data 🔢

Dataset	Nb CLS	No. Training	No. Test
CIFAR10	10	50,000	10,000
CIFAR100	100	50,000	10,000
CUB	200	5,994	5,794
CARS	196	8,144	8,041

One can directly go to ./data/, launch download.sh to download CUB and CARS dataset.

5. Baseline Results 📝

• CUB, CARS are trained and tested with 224 * 224
• CIFAR10, CIFAR100 are trained and tested with 32 * 32

LR	EMA	Arch	CUB	CARS	CIFAR10	CIFAR100
0.05	-	ResNet18	60.74	87.33	95.31	79.13
0.05	TRUE	ResNet18	64.64	87.54	95.62	79.11
0.05	-	ResNet50	57.16	88.87	-	-
0.05	TRUE	ResNet50	63.43	89.27	-	-

EMA improves in most cases Reproducing the above exp with:

bash baseline.sh

6. Pretraining Results 📑

• CUB, CARS are trained and tested with 224 * 224
• Report result with EMA

LR	Pretrained	Arch	CUB	CARS
0.05	-	ResNet18	64.64	87.54
0.005	Inet1K	ResNet18	77.11	88.42
0.05	-	ResNet50	63.43	89.27
0.005	Inet1K	ResNet50	84.47	91.38
0.005	MocoV2	ResNet50	79.01	92.33

Accuracy with DEIT-B and ConvNext-B Models (Resolution 384 * 384)

LR	Pretrained	EMA	Arch	CUB	CARS	CIFAR10	CIFAR100
0.002	Inet1K	-	DEIT-B-384	87.05	92.44	98.33	88.24
0.002	Inet1K	True	DEIT-B-384	87.42	92.87	98.83	90.47
0.002	Inet21K	-	DEIT-B-384	88.87	91.56	98.61	89.73
0.002	Inet21K	True	DEIT-B-384	89.61	92.17	98.99	92.31
0.002	Inet21K + 1K	-	ConvNext-B	91.09	93.35	98.73	92.04
0.002	Inet21K + 1K	True	ConvNext-B	91.22	93.32	98.87	92.40
0.002	Inet21K	-	ConvNext-B	91.49	93.14	98.73	91.76
0.002	Inet21K	True	ConvNext-B	91.58	93.17	98.84	92.14

AURC with DEIT-B and ConvNext-B Models (Resolution 384 * 384)

LR	Pretrained	EMA	Arch	CUB	CARS	CIFAR10	CIFAR100
0.002	Inet1K	-	DEIT-B-384	28.67	16.20	0.78	19.28
0.002	Inet1K	True	DEIT-B-384	27.05	14.44	0.45	13.59
0.002	Inet21K	-	DEIT-B-384	23.76	18.12	0.63	15.77
0.002	Inet21K	True	DEIT-B-384	23.02	15.14	0.50	9.72
0.002	Inet21K + 1K	-	ConvNext-B	19.28	12.92	0.42	10.26
0.002	Inet21K + 1K	True	ConvNext-B	18.79	12.75	0.39	9.14
0.002	Inet21K	-	ConvNext-B	19.78	12.61	0.59	11.08
0.002	Inet21K	True	ConvNext-B	19.41	12.30	0.48	9.78

Pretraining matters Reproducing the above exp with:

cd pretrained_weight/
  bash download.sh
  cd ..
  bash pretraining.sh

7. FixRes Results 📈

• One can refer to [经典论文] Meta的FixRes (NeurIPS 2019)

• CUB, CARS are trained with Inet1K Pretrained weight

• Report result with EMA

Stage 1			Stage 2			Test Size 224*224		Test Size 448*448
LR	Epoch	Train Size	LR	Epoch	Train Size	CUB	CARS	CUB	CARS
0.005	500	224*224	-	-	-	84.47	91.38	73.86	91.58
0.005	500	448*448	-	-	-	78.56	79.23	85.71	91.88
0.005	400	224*224	0.001	100	448*448	82.59	87.46	86.07	92.18

• Training and testing with the same resolution achieves more stable accuracy

• Training on a small resolution then finetuning on larger resolution manage to achieve consistent gain

• Check paperswithcode, these results 86.07 on CUB and 92.18 on CARS are not bad.

Reproducing the above exp with:

  cd pretrained_weight/
  bash download.sh
  cd ..
  bash fix_resolution.sh

8. References 📜

• Fixing the train-test resolution discrepancy
• Deep Residual Learning for Image Recognition
• MOCOV2: Momentum Contrast for Unsupervised Visual Representation Learning
• DeiT III: Revenge of the ViT
• Training data-efficient image transformers & distillation through attention
• ConvNeXt: A ConvNet for the 2020s

关注我们，其他感兴趣的内容

• 公众号文章：极简图像分类，95.58% on CIFAR10 with ResNet18

• [经典论文] Meta的FixRes (NeurIPS 2019)

• [经典论文] 让卷积再次具有平移不变性 (2019 ICML)

• [Pytorch实战] model.eval()究竟干了啥

• [Pytorch实战] 什么是In-place操作

• 关注我们的公众号：AI上分搭子