This project was published in IEEE Robotics and Automation Letters RA-L
[ArXiv Paper]
For a quick overview please visit our website project or watch our video demo depicted below. For further questions, please contact us to [email protected]
This is the implementation of the 360-DFPE, sequential floor plan estimation using only monocular 360-images as input.
Using this REPO, you can:
- Excecute 360-DFPE for single and multiple room scenes in a sequential and non-sequential manner.
- Register multiple 360-layouts with estimated camera poses, assuming an unknown visual odometry scale and missed camera height. For convenience, this registration can be executed isolated from the floor plan estimation pipeline. see Computing metadata
- Evaluate corner and room metrics for floor plan estimation.
- python 3.7.7
- vispy 0.5.3
- numpy 1.18.5
- opencv-python 3.4.3.18
- pandas 1.0.5
The dataset used in this project is our own collected MP3D-FPE dataset, which simulates a handled-camera walking through different rooms scenes rendered by MINOS simulator with MP3D real-world data. Upon this collected data, we annotate floor plan labels, estimated camera poses uisng OpenVSLAM[4], and estimated layout for each key-frame by HorizonNet[3]. Our dataset offers 360-rgb-images, depth maps, ground truth and estimated camera poses, floor plan labels, and estimated layouts.
For convenience, we have prepared a light-set of scenes which can be downloaded by running download_mp3d_fpe.py -h.
All hyperameter settings are stored in a yaml file at ./config/config.yaml. Addtitionally, for practical purposes, we define a data manager class DataManager, which handles the data in the MP3D-FPE dataset, i.e., ground truth information, estimated poses, rgb images, etc. The following is a typical initialization of our system:
from config import read_config
from data_manager import DataManager
from src import DirectFloorPlanEstimation
config_file = "./config/config.yaml"
cfg = read_config(config_file=config_file)
dt = DataManager(cfg)
fpe = DirectFloorPlanEstimation(dt)
...
fpe.scale_recover.esimate_vo_scale()
...For executing 360-DFPE, we have created three main scripts, i.e., main_eval_scene.py, main_eval_list_scenes.py, main_eval_non_seq_approach.py.
python main_eval_scene.py --scene_name 1LXtFkjw3qL_0 --results ./test/python main_eval_scenes_list.py --scene_list ./data/scene_list_pilot.txt --results ./test/python main_eval_non_seq_approach.py --scene_list ./data/scene_list_pilot.txt --results ./test/Note that our formulation rely on monocular estimated camera poses, therefore the real scale of the odometry is missed. Additionally, since every layout geometry is estimated from a monocular inference of HorizonNet[3], the layout scale is also missed. For these reasons, we additionally implemented main_compute_metadata.py, which will compute these missed scales along with some additional information such as ground truth rooms, labels, and room-corners, and the missed scales need for 360-layout registration. For convenience, this script runs isolated from the floor plan estimation pipeline.
python main_compute_metadata.py --scene_list ./data/scene_list_pilot.txt --results ./test/
- Thanks to professor Wei-Chen Chiu for his unvaluable advises in this project.
- The credits of this repo are shared with Yueh-Cheng Liu.
Please cite our paper for any purpose of usage.
@ARTICLE{9772341,
author={Solarte, Bolivar and Liu, Yueh-Cheng and Wu, Chin-Hsuan and Tsai, Yi-Hsuan and Sun, Min},
journal={IEEE Robotics and Automation Letters},
title={360-DFPE: Leveraging Monocular 360-Layouts for Direct Floor Plan Estimation},
year={2022},
volume={7},
number={3},
pages={6503-6510},
doi={10.1109/LRA.2022.3173730}}