DETR has set up a simple end-to-end pipeline for object detection by formulating this task as a set prediction problem, showing promising potential. However, despite the significant progress in improving DETR, this paper identifies a problem of misalignment in the output distribution, which prevents the best-regressed samples from being assigned with high confidence, hindering the model's accuracy. We propose a metric, recall of best-regressed samples, to quantitively evaluate the misalignment problem. Observing its importance, we propose a novel Align-DETR that incorporates a localization precision-aware classification loss in optimization. The proposed loss, IA-BCE, guides the training of DETR to build a strong correlation between classification score and localization precision. We also adopt the mixed-matching strategy, to facilitate DETR-based detectors with faster training convergence while keeping an end-to-end scheme. Moreover, to overcome the dramatic decrease in sample quality induced by the sparsity of queries, we introduce a prime sample weighting mechanism to suppress the interference of unimportant samples. Extensive experiments are conducted with very competitive results reported. In particular, it delivers a 46 (+3.8)% AP on the DAB-DETR baseline with the ResNet-50 backbone and reaches a new SOTA performance of 50.2% AP in the 1x setting on the COCO validation set when employing the strong baseline DINO.
Backbone | Model | Lr schd | box AP | Config | Download |
---|---|---|---|---|---|
R-50 | DINO-4scale | 12e | 50.5 | config | model | log |
R-50 | DINO-4scale | 24e | 51.4 | config | model | log |
We provide the config files for AlignDETR: Align-DETR: Improving DETR with Simple IoU-aware BCE loss.
@misc{cai2023aligndetr,
title={Align-DETR: Improving DETR with Simple IoU-aware BCE loss},
author={Zhi Cai and Songtao Liu and Guodong Wang and Zheng Ge and Xiangyu Zhang and Di Huang},
year={2023},
eprint={2304.07527},
archivePrefix={arXiv},
primaryClass={cs.CV}
}