Vision-based Localization of a Wheeled Mobile Robot with a Stereo Camera on a Pan-tilt Unit

This paper is about a vision-based localization of a wheeled mobile robot (WMR) in an environment that contains multiple artificial landmarks, which are sparsely scattered and at known locations. The WMR is equipped with an on-board stereo camera that can detect the positions and IDs of the landmarks in the stereo image pair. The stereo camera is mounted on a pan-tilt unit that enables rotation of the camera with respect to the mobile robot. The paper presents an approach for calibration of the stereo camera on a pan-tilt unit based on observation of the scene from different views. Calibrated model of the system and the noise model are then used in the extended Kalman filter that estimates the mobile robot pose based on wheel odometry and stereo camera measurements of the landmarks. We assume that the mobile robot drives on a flat surface. In order to enforce this constraint, we transform the localization problem to a two-dimensional space. A short analysis of system observability based on indistinguishable states is also given. The presented models and algorithms were verified and validated in simulation environment.