ROSplat

The Online ROS2-Based Gaussian Splatting-Enabled Visualizer

Shady Gmira

Overview

ROSplat is the first online ROS2-based visualizer that leverages Gaussian splatting to render complex 3D scenes. It is designed to efficiently visualize millions of gaussians by using custom ROS2 messages and GPU-accelerated sorting and rendering techniques. ROSplat also supports data loading from PLY files and integrates with ROS2 tools such as bag recording.

Features

• Real-Time Visualization: Render millions of Gaussian “splats” in real time.
• ROS2 Integration: Built on ROS2 for online data exchange of Gaussians, Images, and IMU data.
• Custom Gaussian Messages: Uses custom message types (SingleGaussian and GaussianArray) to encapsulate properties such as position, rotation, scale, opacity, and spherical harmonics.
• GPU-Accelerated Sorting & Rendering: Offloads sorting and rendering tasks to the GPU.

Setup

This project has been developed and tested on Ubuntu 24.04 LTS using ROS2 Jazzy. It is important to note that this project performs significantly worse without a NVIDIA graphics card.

Dependencies

• Mandatory: ROS2 (tested on ROS2 Jazzy)
• Optional (for GPU-based Sorting):
  • cupy (or a variant matching your CUDA version)
  • torch (if using PyTorch for GPU sorting)

To install these dependencies, run:

pip install -r requirements.txt

Docker-Based Setup

If you prefer to use Docker, the project includes a setup script under the docker directory.

cd docker
./run_docker.sh -h    # To display help and usage instructions
./run_docker.sh -bu   # To build the Docker image and launch the container using docker-compose

This method ensures a consistent environment and avoids dependency issues.

Building the Gaussian Messages

ROSplat defines two custom ROS2 messages to handle Gaussian data, located in the gaussian_interface/msg folder.

Note: The Gaussian messages are based on the original Gaussian Splatting implementation by graphdeco-inria.

Message Definitions

1. SingleGaussian.msg

geometry_msgs/msg/Point xyz
geometry_msgs/msg/Quaternion rotation
float32 opacity
geometry_msgs/msg/Vector3 scale
float32[] spherical_harmonics

2. GaussianArray.msg

gaussian_interface/SingleGaussian[] gaussians

Building the Messages

a) Build your workspace using colcon:

colcon build --packages-select gaussian_interface

b) Source your workspace:

. install/setup.bash

Important: Depending on your shell, you might need to adjust these commands.

Usage

Once the Gaussian messages are built, you can launch the visualizer from the project's root directory:

cd src
python main.py

Contributions

Contributions and feedback are welcome!

Acknowledgments

I'm glad to have worked on such a challenging topic and grateful for the invaluable advice and support I received throughout this project.

Special thanks to Qihao Yuan and Kailai Li for their guidance and encouragement as well as the constructive feedback that helped shape this work.

This project was additionally influenced by limacv 's implementation of the GaussianSplattingViewer repository.

Contact

For questions or further information, please email: shady.gmira[at]gmail.com