Abstract
Tasks such as autonomous navigation, 3D reconstruction, and object
recognition near the water surfaces are crucial in marine robotics
applications. However, challenges arise due to dynamic disturbances, e.g.,
light reflections and refraction from the random air-water interface, irregular
liquid flow, and similar factors, which can lead to potential failures in
perception and navigation systems. Traditional computer vision algorithms
struggle to differentiate between real and virtual image regions, significantly
complicating tasks. A virtual image region is an apparent representation formed
by the redirection of light rays, typically through reflection or refraction,
creating the illusion of an object's presence without its actual physical
location. This work proposes a novel approach for segmentation on real and
virtual image regions, exploiting synthetic images combined with
domain-invariant information, a Motion Entropy Kernel, and Epipolar Geometric
Consistency. Our segmentation network does not need to be re-trained if the
domain changes. We show this by deploying the same segmentation network in two
different domains: simulation and the real world. By creating realistic
synthetic images that mimic the complexities of the water surface, we provide
fine-grained training data for our network (MARVIS) to discern between real and
virtual images effectively. By motion & geometry-aware design choices and
through comprehensive experimental analysis, we achieve state-of-the-art
real-virtual image segmentation performance in unseen real world domain,
achieving an IoU over 78% and a F1-Score over 86% while ensuring a small
computational footprint. MARVIS offers over 43 FPS (8 FPS) inference rates on a
single GPU (CPU core). Our code and dataset are available here
https://github.com/jiayi-wu-umd/MARVIS.