PSENet

Shape robust text detection with progressive scale expansion network

Abstract

Scene text detection has witnessed rapid progress especially with the recent development of convolutional neural networks. However, there still exists two challenges which prevent the algorithm into industry applications. On the one hand, most of the state-of-art algorithms require quadrangle bounding box which is in-accurate to locate the texts with arbitrary shape. On the other hand, two text instances which are close to each other may lead to a false detection which covers both instances. Traditionally, the segmentation-based approach can relieve the first problem but usually fail to solve the second challenge. To address these two challenges, in this paper, we propose a novel Progressive Scale Expansion Network (PSENet), which can precisely detect text instances with arbitrary shapes. More specifically, PSENet generates the different scale of kernels for each text instance, and gradually expands the minimal scale kernel to the text instance with the complete shape. Due to the fact that there are large geometrical margins among the minimal scale kernels, our method is effective to split the close text instances, making it easier to use segmentation-based methods to detect arbitrary-shaped text instances. Extensive experiments on CTW1500, Total-Text, ICDAR 2015 and ICDAR 2017 MLT validate the effectiveness of PSENet. Notably, on CTW1500, a dataset full of long curve texts, PSENet achieves a F-measure of 74.3% at 27 FPS, and our best F-measure (82.2%) outperforms state-of-art algorithms by 6.6%. The code will be released in the future.

Results and models

CTW1500

Method	Backbone	Pretrained Model	Training set	Test set	#epochs	Test size	Precision	Recall	Hmean	Download
PSENet	ResNet50	-	CTW1500 Train	CTW1500 Test	600	1280	0.7705	0.7883	0.7793	model | log
PSENet_r50-oclip	ResNet50-oCLIP	-	CTW1500 Train	CTW1500 Test	600	1280	0.8483	0.7636	0.8037	model | log

ICDAR2015

Method	Backbone	Pretrained Model	Training set	Test set	#epochs	Test size	Precision	Recall	Hmean	Download
PSENet	ResNet50	-	IC15 Train	IC15 Test	600	2240	0.8396	0.7636	0.7998	model | log
PSENet_r50-oclip	ResNet50-oCLIP	-	IC15 Train	IC15 Test	600	2240	0.8895	0.8098	0.8478	model | log

Citation

@inproceedings{wang2019shape,
  title={Shape robust text detection with progressive scale expansion network},
  author={Wang, Wenhai and Xie, Enze and Li, Xiang and Hou, Wenbo and Lu, Tong and Yu, Gang and Shao, Shuai},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={9336--9345},
  year={2019}
}