FisheyeDepth

FisheyeDepth: A Real Scale Self-Supervised Depth Estimation Model for Fisheye Camera

Motivation:

(a) Fisheye cameras offer a wide FOV and advanced geometric properties, making them well-suited for robotic vision tasks.
(b) The significant distortion in fisheye images poses a major challenge for depth estimation networks in achieving pixel-wise geometric consistency between consecutive frames.
(c) Depth estimation models often neglect the real motion scale during training, limiting their applicability in real-world robotic systems.

Framework:

Self-supervised framework for the fisheye camera.
(a) We propose a fisheye projection scheme based on the camera model to eliminate distortion in training, which greatly improves the accuracy of depth estimation.
(b) We incorporate real-scale poses during the training process, which renders the model suitable for real-world robotic interactions and navigation tasks.
(c) We devise a multi-channel output module that performs adaptive feature fusion across multiple scales, and this ensures robust depth predictions across different scenes.

Results:

Depth Estimation Results – KITTI-360:

Depth Estimation Results – Real Scene:

More visualization results can be viewed through the Demo Video

Citations:

If you find FisheyeDepth useful in your research or applications, please consider giving us a star 🌟 and citing it.

@misc{zhao2024fisheyedepthrealscaleselfsupervised,
      title={FisheyeDepth: A Real Scale Self-Supervised Depth Estimation Model for Fisheye Camera}, 
      author={Guoyang Zhao and Yuxuan Liu and Weiqing Qi and Fulong Ma and Ming Liu and Jun Ma},
      year={2024},
      eprint={2409.15054},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2409.15054}, 
}