datasets readme

.gitignore

@@ -1 +1,2 @@
 ./demo*
+./logs/*

README.md

@@ -1,58 +1,97 @@
-# Coex
+# CoEx
 
 PyTorch implementation of our paper: 
 
-### **Correlate-and-Excite: Real-Time Stereo Matching via Guided Cost Volume Excitation**  
-*Authors: Antyanta Bangunharcana, Jae Won Cho, Seokju Lee, In So Kweon, Kyung-Soo Kim, Soohyun Kim*
+
+**Correlate-and-Excite: Real-Time Stereo Matching via Guided Cost Volume Excitation**  
+*Authors: Antyanta Bangunharcana, Jae Won Cho, Seokju Lee, In So Kweon, Kyung-Soo Kim, Soohyun Kim*  
+IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2021
+
+\[[Project page](https://antabangun.github.io/projects/CoEx/)\]
 
 We propose a Guided Cost volume Excitation (GCE) and top-k soft-argmax disparity regression for real-time and accurate stereo matching. 
 
+## Contents
+- [Installation](#Installation)
+- [Datasets](#datasets)
+    - [Data for demo](#data-for-demo)
+    - [If you want to re-train the model](#if-you-want-to-re-train-the-model)
+- [Demo on KITTI raw data](#demo-on-kitti-raw-data)
+    - [Model zoo](#model-zoo)
+- [Re-training the models](#re-training-the-models)
 
-## Installation
+##Installation
 
 We recommend using [conda](https://www.anaconda.com/distribution/) for installation: 
-
 ```shell
 conda env create -f environment.yml
 ```
-
+Then activate the newly created env
+```shell
+conda activate coex
+```
 
 ## Datasets
+
 ```
 data
-├── KITTI
-│   ├── kitti_2012
-│   │   └── data_stereo_flow
-│   ├── kitti_2015
-│   │   └── data_scene_flow
-└── SceneFlow
-    ├── Driving
-    │   ├── disparity
-    │   └── frames_finalpass
-    ├── FlyingThings3D
-    │   ├── disparity
-    │   └── frames_finalpass
-    ├── Monkaa
-    │   ├── disparity
-    │   └── frames_finalpass
-    ├── kitti12
-    │   ├── disparity
-    │   └── frames_finalpass
-    └── kitti15
-        ├── disparity
-        └── frames_finalpass
+└── datasets
+    ├── KITTI_raw
+    |   ├── 2011_09_26
+    |   │   ├── 2011_09_26_drive_0001_sync
+    |   │   ├── 2011_09_26_drive_0002_sync
+    |   |       :
+    |   |
+    |   ├── 2011_09_28
+    |   │   ├── 2011_09_28_drive_0001_sync
+    |   │   └── 2011_09_28_drive_0002_sync
+    |   |       :
+    |   |   :    
+    |
+    └── SceneFlow
+        ├── driving
+        │   ├── disparity
+        │   └── frames_finalpass
+        ├── flyingthings3d_final
+        │   ├── disparity
+        │   └── frames_finalpass
+        ├── monkaa
+        │   ├── disparity
+        │   └── frames_finalpass
+        ├── kitti12
+        │   ├── testing
+        │   └── training
+        └── kitti15
+            ├── testing
+            └── training
 ```
 
+### Data for demo
+
+For a demo of our code on the KITTI dataset, download the "\[synced+rectified data\]" from [raw KITTI data](http://www.cvlibs.net/datasets/kitti/raw_data.php). Unzip and place the extracted folders following the above directory tree. 
+
+### If you want to re-train the models
+**Sceneflow dataset**  
+Download the *finalpass* data of the [Sceneflow dataset](https://lmb.informatik.uni-freiburg.de/resources/datasets/SceneFlowDatasets.en.html) as well as the *Disparity* data.
+
+**KITTI 2015**  
+Download [kitti15](http://www.cvlibs.net/datasets/kitti/eval_scene_flow.php?benchmark=stereo) dataset, and unzip data_scene_flow.zip, rename it as kitti15, and move it into SceneFlow directory as shown in the tree above.
+
+**KITTI 2012**  
+Download [kitti12](http://www.cvlibs.net/datasets/kitti/eval_stereo_flow.php?benchmark=stereo) dataset. Unzip data_stereo_flow.zip, rename it as kitti12, and move it into SceneFlow directory as shown in the tree above.
+
+The paths where the code can find the dataset can be modified inside the [config files](/configs/stereo/cfg_coex.yaml), but make sure the directory names of driving, flyingthings3d_final, monkaa.
+
 ## Demo on KITTI raw data
 ### Model zoo
 
 
-## Training your own model
+## Re-training the models
 
 
 ## Citation
 
-If you find our work useful in your research, please consider citing our paper:
+If you find our work useful in your research, please consider citing our paper
 
 ## Acknowledgements
 

environment.yml

@@ -6,14 +6,13 @@ channels:
   - anaconda
   - defaults
 dependencies:
-  - python>=3.7
-  - anaconda
+  - python=3.7
   - pip
   - numpy
-  - pytorch
-  - torchvision
+  - pytorch=1.9.0
+  - torchvision=0.10.0
   - cudatoolkit=11.1
-  - pytorch-lightning
+  - pytorch-lightning=1.4.0
   - ruamel.yaml
   - pillow
   - scikit-image

models/stereo/submodules/util_conv.py

@@ -1,17 +1,13 @@
 #!/usr/bin/env python
 from __future__ import print_function
-import os
-import numpy as np
-from typing import Callable, Any, Optional, List
+from typing import Callable, Optional, List
 import torch
 from torch import Tensor
 import torch.nn as nn
-import torch.nn.functional as F
-from torchvision.ops import DeformConv2d
-from torch.autograd import Variable
 
 import pdb
 
+
 class BasicConv(nn.Module):
 
     def __init__(self, in_channels, out_channels, deconv=False, is_3d=False, bn=True, relu=True, **kwargs):
@@ -41,33 +37,6 @@ def forward(self, x):
         return x
 
 
-class DeformConv(nn.Module):
-    def __init__(self, inp_c, out_c, kernel_size=3, stride=1, padding=1, bias=False):
-        super(DeformConv, self).__init__()
-        self.offset = nn.Conv2d(inp_c, 2*3*3,
-                                kernel_size=kernel_size,
-                                stride=stride,
-                                padding=padding)
-        self.mask = nn.Sequential(
-            nn.Conv2d(inp_c, 1*3*3,
-                      kernel_size=kernel_size,
-                      stride=stride,
-                      padding=padding),
-            nn.Sigmoid())
-        self.conv = DeformConv2d(inp_c, out_c,
-                                 kernel_size=kernel_size,
-                                 stride=stride,
-                                 padding=padding,
-                                 bias=bias)
-        # self.bnrelu = nn.Sequential(nn.BatchNorm2d(out_c), nn.ReLU6())
-
-    def forward(self, x):
-        offset = self.offset(x)
-        mask = self.mask(x)
-        x = (self.conv(x, offset, mask))
-        return x
-
-
 class Conv2x(nn.Module):
 
     def __init__(self, in_channels, out_channels, deconv=False, is_3d=False, concat=True, keep_concat=True, bn=True, relu=True, keep_dispc=False):