Accurate Global Localization Using Visual Odometry and Digital Maps on Urban Environments

Over the past few years, advanced driver-assistance systems (ADASs) have become a key element in the research and development of intelligent transportation systems (ITSs) and particularly of intelligent vehicles. Many of these systems require accurate global localization information, which has been traditionally performed by the Global Positioning System (GPS), despite its well-known failings, particularly in urban environments. Different solutions have been attempted to bridge the gaps of GPS positioning errors, but they usually require additional expensive sensors. Vision-based algorithms have proved to be capable of tracking the position of a vehicle over long distances using only a sequence of images as input and with no prior knowledge of the environment. This paper describes a full solution to the estimation of the global position of a vehicle in a digital road map by means of visual information alone. Our solution is based on a stereo platform used to estimate the motion trajectory of the ego vehicle and a map-matching algorithm, which will correct the cumulative errors of the vision-based motion information and estimate the global position of the vehicle in a digital road map. We demonstrate our system in large-scale urban experiments reaching high accuracy in the estimation of the global position and allowing for longer GPS blackouts due to both the high accuracy of our visual odometry estimation and the correction of the cumulative error of the map-matching algorithm. Typically, challenging situations in urban environments such as nonstatic objects or illumination exceeding the dynamic range of the cameras are shown and discussed.

[1]  Junie McCree The travelling salesman , 1913 .

[2]  A. C. Aitken IV.—On Least Squares and Linear Combination of Observations , 1936 .

[3]  S. Laurila Electronic surveying and navigation , 1976 .

[4]  Hans P. Moravec Obstacle avoidance and navigation in the real world by a seeing robot rover , 1980 .

[5]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[6]  Franc Solina,et al.  Errors in stereo due to quantization , 1985 .

[7]  Steven D. Blostein,et al.  Error Analysis in Stereo Determination of 3-D Point Positions , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[9]  Zhengyou Zhang,et al.  Estimation of Displacements from Two 3-D Frames Obtained From Stereo , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  A. Kornhauser,et al.  An Introduction to Map Matching for Personal Navigation Assistants , 1998 .

[11]  Manuel Mazo,et al.  Automation of an Industrial Fork Lift Truck, Guided by Artificial Vision in Open Environments , 1998, Auton. Robots.

[12]  Peter Meer,et al.  Optimal rigid motion estimation and performance evaluation with bootstrap , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[13]  Amnon Shashua,et al.  Trajectory Triangulation: 3D Reconstruction of Moving Points from a Monocular Image Sequence , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Amnon Shashua,et al.  A robust method for computing vehicle ego-motion , 2000, Proceedings of the IEEE Intelligent Vehicles Symposium 2000 (Cat. No.00TH8511).

[15]  J. Greenfeld MATCHING GPS OBSERVATIONS TO LOCATIONS ON A DIGITAL MAP , 2002 .

[16]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[17]  Washington Y. Ochieng,et al.  MAP-MATCHING IN COMPLEX URBAN ROAD NETWORKS , 2009, Revista Brasileira de Cartografia.

[18]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  A. Shashua,et al.  Pedestrian detection for driving assistance systems: single-frame classification and system level performance , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[20]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[21]  James R. Bergen,et al.  Visual odometry , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[22]  Mei Han,et al.  Reconstruction of a Scene with Multiple Linearly Moving Objects , 2004, International Journal of Computer Vision.

[23]  David Nistér,et al.  A Minimal Solution to the Generalised 3-Point Pose Problem , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[24]  Luis Magdalena,et al.  VIRTUOUS: vision-based road transportation for unmanned operation on urban-like scenarios , 2004, IEEE Transactions on Intelligent Transportation Systems.

[25]  Kurt Konolige,et al.  Real-time Localization in Outdoor Environments using Stereo Vision and Inexpensive GPS , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[26]  Michel Dhome,et al.  Real Time Localization and 3D Reconstruction , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[27]  Gérard G. Medioni,et al.  3D Reconstruction of Background and Objects Moving on Ground Plane Viewed from a Moving Camera , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[28]  Javier Civera,et al.  Unified Inverse Depth Parametrization for Monocular SLAM , 2006, Robotics: Science and Systems.

[29]  James R. Bergen,et al.  Visual odometry for ground vehicle applications , 2006, J. Field Robotics.

[30]  Larry Matthies,et al.  Two years of Visual Odometry on the Mars Exploration Rovers: Field Reports , 2007 .

[31]  Ignacio Parra,et al.  Visual odometry for road vehicles—feasibility analysis , 2007 .

[32]  Kurt Konolige,et al.  Large-Scale Visual Odometry for Rough Terrain , 2007, ISRR.

[33]  Larry H. Matthies,et al.  Two years of Visual Odometry on the Mars Exploration Rovers , 2007, J. Field Robotics.

[34]  Ian D. Reid,et al.  Mapping Large Loops with a Single Hand-Held Camera , 2007, Robotics: Science and Systems.

[35]  Robert B. Noland,et al.  Current map-matching algorithms for transport applications: State-of-the art and future research directions , 2007 .

[36]  Ignacio Parra,et al.  Combination of Feature Extraction Methods for SVM Pedestrian Detection , 2007, IEEE Transactions on Intelligent Transportation Systems.

[37]  Navid Nourani-Vatani,et al.  IMU aided 3D visual odometry for car-like vehicles , 2008, ICRA 2008.

[38]  Kurt Konolige,et al.  CenSurE: Center Surround Extremas for Realtime Feature Detection and Matching , 2008, ECCV.

[39]  Yu Meng,et al.  Tight integration of digital map and in-vehicle positioning unit for car navigation in urban areas , 2008, Wuhan University Journal of Natural Sciences.

[40]  Ignacio Parra,et al.  3D Visual Odometry for Road Vehicles , 2008, J. Intell. Robotic Syst..

[41]  Roland Siegwart,et al.  Appearance-Guided Monocular Omnidirectional Visual Odometry for Outdoor Ground Vehicles , 2008, IEEE Transactions on Robotics.

[42]  Ignacio Parra,et al.  Robust visual odometry for vehicle localization in urban environments , 2009, Robotica.

[43]  Ignacio Parra,et al.  An Experimental Study on Pitch Compensation in Pedestrian-Protection Systems for Collision Avoidance and Mitigation , 2009, IEEE Transactions on Intelligent Transportation Systems.

[44]  M. Dhome,et al.  Towards geographical referencing of monocular SLAM reconstruction using 3D city models: Application to real-time accurate vision-based localization , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Ignacio Parra,et al.  Perception advances in outdoor vehicle detection for automatic cruise control , 2010, Robotica.

[46]  Paul Newman,et al.  Real-time bounded-error pose estimation for road vehicles using vision , 2010, 13th International IEEE Conference on Intelligent Transportation Systems.

[47]  Akira Ishii,et al.  Correcting angle of visual odometry system by fusing monocular and stereo methods in untextured dynamic environment , 2010, 2010 10th International Symposium on Communications and Information Technologies.

[48]  Ashwin P. Dani,et al.  Structure and motion estimation of a moving object using a moving camera , 2010, Proceedings of the 2010 American Control Conference.

[49]  Javier Ibanez Guzman,et al.  Vehicle to vehicle communications applied to road intersection safety, field results , 2010, 13th International IEEE Conference on Intelligent Transportation Systems.

[50]  Yaser Sheikh,et al.  3D Reconstruction of a Moving Point from a Series of 2D Projections , 2010, ECCV.

[51]  Ignacio Parra,et al.  Error Analysis in a Stereo Vision-Based Pedestrian Detection Sensor for Collision Avoidance Applications , 2010, Sensors.

[52]  Javier Ibanez Guzman,et al.  Accurate visual odometry from a rear parking camera , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[53]  Warren E. Dixon,et al.  Structure estimation of a moving object using a moving camera: An unknown input observer approach , 2011, IEEE Conference on Decision and Control and European Control Conference.