This is unofficial implementation of "AutoRF: Learning 3D Object Radiance Fields from Single View Observations", which performs implicit neural reconstruction, manipulation and scene composition for 3D object. In this repo, we use KITTI dataset.
Download KITTI data and here we only use image data
└── DATA_DIR
├── training <-- training data
| ├── image_2
| ├── label_2
| ├── calib
Run the preprocess scripts, which produce instance mask using pretrained PointRend model.
python scripts/preproc.py
After this, you will have a certain directory which contains the image, mask and 3D anotation of each instance.
└── DATA_DIR
├── training
| ├── nerf
| ├── 0000008_01_patch.png
| ├── 0000008_01_mask.png
| ├── 0000008_01_label.png
Run the following sciprts to train a nerf model
python src/train.py
After training for serveral iterations (enough is ok), you can find the checkpoint file in the ``output'' folder, and then you can perform scene rendering by running
python src/train.py --demo
You can adjust the manipulaion function (in kitti.py) by your self, here I only provide the camera pushing/pulling and instance rotation.