Multiple fisheye cameras improve visual inertial pose tracking accuracy
MFVINS: Multiple Fisheye Camera-Based Visual Inertial System
RoboticsComputer Vision and Pattern Recognition
Summary
Position tracking with one camera and an inertial sensor can struggle when things block the view or there’s little detail to see. The authors created a system called MFVINS that uses several fisheye cameras working together with an inertial sensor to track movement more accurately. They developed a special way to find matching points in the images from multiple fisheye cameras and fix distortions from the curved lenses. This helps keep track of position better even when parts of the view are blocked or look plain. They tested their method to show it runs in real-time and works better than methods using just one camera.
What this means in practice
- •For robotics engineers: Implement robust real-time localization and mapping for robots navigating in challenging environments with occlusion and low texture using multiple fisheye cameras and inertial sensing.
- •For augmented reality developers: Develop AR devices that maintain accurate position tracking in varied lighting and occluded scenes by combining multiple fisheye cameras with inertial measurement units.
Authors
Eunseong Jang, YuJin Chung, Sang Jun Lee, Jihyun Yoon, HyungGi Jo
Abstract
A simultaneous localization and mapping (SLAM) method using a monocular camera and a low-cost inertial measurement unit (IMU) sensor is an effective way to fulfill a low-cost sensor configuration. Using this sensor configuration, visual-inertial system (VINS) focuses on fusing data from a camera and an IMU sensor to estimate the six degrees-of-freedom (DOF) of the sensor pose. Typically, VINS uses only a single camera as visual input, which lead to problems such as error accumulation due to occlusion, various illumination, and textureless environments. In this paper, we propose a new multiple fisheye camera-based visual-inertial system called MFVINS. We present an IMU-aided FAST feature tracker for multiple cameras that enables efficient extraction and robust matching of local features. Then, the proposed method filters out outliers caused by fisheye distortion on the normalized image plane. Subsequently, a new reprojection error with physical validity constraints is proposed for bundle adjustment using learning-based depth estimation. The proposed method is applied to various scenarios, and its effectiveness is demonstrated by comparing previous VINS methods. In particular, MFVINS is implemented in real-time process to leverage the advantages of using multiple cameras -- robustness against occlusion and textureless regions -- while reducing the computational burden.