Hiring research interns for visual tracking, segmentation and neural architecture search projects: [email protected]

TracKit

This is a toolkit for video object tracking and segmentation.

News

💥 Implementation of our tracking and segmentation work, based on Pytorch and TensorRT.

💥 The initial version is released, including Ocean and SiamDW(CVPR2019).

💥 We provide a TensorRT implementation, running at 1.5~2.5 times faster than pytorch version (e.g. 149fps/68fps for video twinnings, see details).

Note: We focus on providing an easy-to-follow code for research on video object tracking and segmentation task. The code will be continuously optimized. You may pull requests to help us build this repo.

Trackers

Ocean

[Paper] [Raw Results] [Training and Testing] [Demo]
Official implementation of the Ocean tracker. Ocean proposes a general anchor-free based tracking framework. It includes a pixel-based anchor-free regression network to solve the weak rectification problem of RPN, and an object-aware classification network to learn robust target-related representation. Moreover, we introduce an effective multi-scale feature combination module to replace heavy result fusion mechanism in recent Siamese trackers. This work also serves as the baseline model of OceanPlus. An additional TensorRT toy demo is provided in this repo.

OceanPlus

Paper and Code will be released soon.

• Advantages: only several lines of code (core part) on Ocean, easy to implement.

• VOT2020 performances

Models	Baseline	Realtime
Offline	0.444	0.436
Online	0.500	0.484

SiamDW

[Paper] [Raw Results] [Training and Testing] [Demo]
SiamDW is one of the pioneering work using deep backbone networks for Siamese tracking framework. Based on sufficient analysis on network depth, output size, receptive field and padding mode, we propose guidelines to build backbone networks for Siamese tracker. Several deeper and wider networks are built following the guidelines with the proposed CIR module.

How To Start

• Tutorial for Ocean

  Follow Ocean [Training and Testing] tutorial

• Tutorial for SiamDW

  Follow SiamDW [Training and Testing] tutorial

Structure

• experiments: training and testing settings
• demo: figures for readme
• dataset: testing dataset
• data: training dataset
• lib: core scripts for all trackers
• snapshot: pre-trained models
• pretrain: models trained on ImageNet (for training)
• tutorials: guidelines for training and testing
• tracking: training and testing interface

$TrackSeg
|—— experimnets
|—— lib
|—— snapshot
  |—— xxx.model/xxx.pth
|—— dataset
  |—— VOT2019.json 
  |—— VOT2019
     |—— ants1...
  |—— VOT2020
     |—— ants1...
|—— ...

ToDO

⚓ Release paper and code for OceanPlus.

Citation

If any part of our paper or code helps your work, please generouslly cite our work:

@InProceedings{SiamDW_2019_CVPR,
author = {Zhang, Zhipeng and Peng, Houwen},
title = {Deeper and Wider Siamese Networks for Real-Time Visual Tracking},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}
} 

@InProceedings{TVOS_2020_CVPR,
author = {Zhang, Yizhuo and Wu, Zhirong and Peng, Houwen and Lin, Stephen},
title = {A Transductive Approach for Video Object Segmentation},
booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2020}
}

@article{Ocean_2020,
  title={Ocean: Object-aware Anchor-free Tracking},
  author={Zhipeng Zhang, Houwen Peng, Jianlong Fu, Bing Li, Weiming Hu},
  journal={arXiv preprint arXiv:2006.10721},
  year={2020}
}

References

[1] Bhat G, Danelljan M, et al. Learning discriminative model prediction for tracking. ICCV2019.
[2] Chen, Kai and Wang, et.al. MMDetection: Open MMLab Detection Toolbox and Benchmark.

Contributors

• Zhipeng Zhang
• Houwen Peng

🎯 Further discussion anbout our paper and code: [email protected]