OmniNxt: A Fully Open-source and Compact Aerial Robot with Omnidirectional Visual Perception

IROS 2024 Oral
Peize Liu, Chen Feng^†, Yang Xu, Yan Ning, Hao Xu^†, and Shaojie Shen

HKUST Aerial Robotics Group   
^†Corresponding Authors

News

• [30/06/2024]: OmniNxt is accepted to IROS 2024.
• [15/08/2024] V0.1 Release

Hardware

You can access our latest design by using Fusion360.

Projest link: https://a360.co/3vK6dJd

Access code: hkustUAV

BOM

All our components can be purchased at Taobao

Component	Specification	Link
Quad-Fisheye Camera Set
OAK-FFC-4P	Camera control board	Official / Taobao
B0335 (OV9782)	Camera modules (210-degree fisheye lens)	Official / Taobao
Camera cables	Customized	Camera cable SCH
Onboard computer
Jetson Orin Nx
Jetson Orin Nx carrier board		DM
Jetson Orin Nx radiator
Jeston Orin Network card
2230 SSD
Intel AX200
Flight platforms (6S)
Nxt-FC	Project Page	MicoAir / Taobao
Flight frame	Customized	Fusion360
Protector	Customized / Oddity RC XI35 protector	Fusion360 / Oddity RC
Motors	2204 1750KV/ 1804 2450KV
6S Battery

Camera cable SCH

---

---

System Settings

Multi-fisheye camera set

Camera hardware preparation and ROS wrap driver see oak_ffc_4p

Calibrate this camera set see quater-calib

Jetson Orin

Our Jetson Info:

We suggest you follow these settings to avoid conflict (especially CUDA and tensorRT).

Nxt-FC

We open-source a coin-size PX4-based flight controller see NXT-FC

Follow the Setting up Nxt-FC part to configure your flight controller.

Runtime setup

Our Omni-VINS and Omni-Depth are developed from D2SLAM. Since building the docker images on the local host (Jetson Orin) usually takes a long time, we suggest you pull the images from the docker-hub.

the docker images structure is as follows:

Step 1 Clone the repository into your local machine

git clone --branch pr_fix_main https://github.com/HKUST-Aerial-Robotics/D2SLAM.git

Step 2 Configure your quad-cam parameters and VINS parameters following our template

cd ./D2SLAM/config/quadcam_drone_nxt_tmp

if you have already calibrated your quad fisheye camera set, you can simply replace the files with the same name, which includes:

1. stereo_calib_n_m_240_320.yaml (n and m is the camera number; used in Omni-Depth).
2. fisheye_cams.yaml. (used in Omni-VINS)

Step 3 Configure ./start_docker.sh script with your local environment.

./start_docker.sh is under the ./D2SLAM directory.

Modify the following parameters with the absolute path under your local host environment.

1. DATA_SET (where the data set is) (Optional)

Then run with the following command under the D2SLAM directory(very important, this will map your D2SLAM dir into the container)：

./start_docker.sh 1

Step 4 Launch algorithm modules

run all algorithm modules together (Omni-VINS & Omni-Depth)

## you should under /root/swarm_ws/
source ./devel/setup.bash
roslaunch d2vins quadcam.laucnh

Only launch Omni-VINS, please remove the nodes in the red box

and then

source ./devel/setup.bash
roslaunch d2vins quadcam.laucnh

Only launch Omni-Depth

source ./devel/setup.bash
roslaunch quadcam_depth_est depth-node.launch

[Notice] If you are running Omni-VINS and Omni-Depth for the first time, the initialization time for these two modules would be a little bit long as these two modules would generate the inference engine.

Almost done

If everything works well, you will see

which means Omni-VINS(D2VINS) initialized normally.

And

for Omni-Depth.

Other settings

OmniNxt px4 parameter:

PX4 Controller

We highly suggest you refer to Fast-Lab's PX4 controller PX4-Control

We also provide our yaw-rotation-free version of Fast-Lab's PX4 controller PX4-Control.

Please cite ZJU-Fast-Lab if these modules are useful for your research and project.

Flight planer

We modified some of the strategies in the ego-planner. Our version is here ego-planner-omni-modify

Please cite ZJU-Fast-Lab's ego-planner if these modules are useful for your research and project.

Trouble shooting

Reference