ADE20K semantic segmentation with TransNeXt

Getting started

This repository is the official PyTorch implementation of TransNeXt for ADE20K semantic segmentation.

Our code is built on MMSegmentation. The UPerNet method is built on MMSegmentation version 0.30.0, while the Mask2Former method is built on MMSegmentation version 1.0.0.

Since MMSegmentation is no longer compatible with the previous version of the configuration file format after 1.0.0, different environments need to be built for the two methods. The requirements.txt can be found in their respective folders.

Model Zoo

ADE20K semantic segmentation results using the UPerNet method:

Backbone	Pretrained Model	Crop Size	Lr Schd	mIoU	mIoU (ms+flip)	#Params	Download	Config	Log
TransNeXt-Tiny	ImageNet-1K	512x512	160K	51.1	51.5/51.7	59M	model	config	log
TransNeXt-Small	ImageNet-1K	512x512	160K	52.2	52.5/52.8	80M	model	config	log
TransNeXt-Base	ImageNet-1K	512x512	160K	53.0	53.5/53.7	121M	model	config	log

• In the context of multi-scale evaluation, TransNeXt reports test results under two distinct scenarios: interpolation and extrapolation of relative position bias.

ADE20K semantic segmentation results using the Mask2Former method:

Backbone	Pretrained Model	Crop Size	Lr Schd	mIoU	#Params	Download	Config	Log
TransNeXt-Tiny	ImageNet-1K	512x512	160K	53.4	47.5M	model	config	log
TransNeXt-Small	ImageNet-1K	512x512	160K	54.1	69.0M	model	config	log
TransNeXt-Base	ImageNet-1K	512x512	160K	54.7	109M	model	config	log

How to use

The code & tutorial for the UPerNet method are >> here <<

The code & tutorial for the Mask2Former method are >> here <<

Acknowledgement

The released script for Object Detection with TransNeXt is built based on the MMSegmentation and timm library.

License

This project is released under the Apache 2.0 license. Please see the LICENSE file for more information.

Citation

If you find our work helpful, please consider citing the following bibtex. We would greatly appreciate a star for this project.

@InProceedings{shi2023transnext,
  author    = {Dai Shi},
  title     = {TransNeXt: Robust Foveal Visual Perception for Vision Transformers},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  month     = {June},
  year      = {2024},
  pages     = {17773-17783}
}