Visual Navigation in Unknown Environments

Navigation in mobile robotics involves two tasks, keeping track of the robot's position and moving according to a control strategy. In addition, when no prior knowledge of the environment is available, the problem is even more difficult, as the robot has to build a map of its surroundings as it moves. These three problems ought to be solved in conjunction since they depend on each other. This thesis is about simultaneously controlling an autonomous vehicle, estimating its location and building the map of the environment. The main objective is to analyse the problem from a control theoretical perspective based on the EKF-SLAM implementation. The contribution of this thesis is the analysis of system's properties such as observability, controllability and stability, which allow us to propose an appropriate navigation scheme that produces well-behaved estimators, controllers, and consequently, the system as a whole. We present a steady state analysis of the SLAM problem, identifying the conditions that lead to partial observability. It is shown that the effects of partial observability appear even in the ideal linear Gaussian case. This indicates that linearisation alone is not the only cause of SLAM inconsistency, and that observability must be achieved as a prerequisite to tackling the effects of linearisation. Additionally, full observability is also shown to be necessary during diagonalisation of the covariance matrix, an approach often used to reduce the computational complexity of the SLAM algorithm, and which leads to full controllability as we show in this work. Focusing specifically on the case of a system with a single monocular camera, we present an observability analysis using the nullspace basis of the stripped observability matrix. The aim is to get a better understanding of the well known intuitive behaviour of this type of systems, such as the need for triangulation to features from different positions in order to get accurate relative pose estimates between vehicle and camera. Through characterisation the unobservable directions in monocular SLAM, we are able to identify the vehicle motions required to maximise the number of observable states in the system. When closing the control loop of the SLAM system, both the feedback controller and the estimator are shown to be asymptotically stable. Furthermore, we show that the tracking error does not influence the estimation performance of a fully observable system and viceversa, that control is not affected by the estimation. Because of this, a higher level motion strategy is required in order to enhance estimation, specially needed while performing SLAM with a single camera. Considering a real-time application, we propose a control strategy to optimise both the localisation of the vehicle and the feature map by computing the most appropriate control actions or movements. The actions are chosen in order to maximise an information theoretic metric. Simulations and real-time experiments are performed to demonstrate the feasibility of the proposed control strategy.

[1]  Anil K. Jain,et al.  Mobile robot localization via classification of multisensor maps , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[2]  H. Durrant-Whyte,et al.  A closed form solution to the single degree of freedom simultaneous localisation and map building (SLAM) problem , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[3]  Rui P. Rocha,et al.  Cooperative Multi-Robot Systems A study of Vision-based 3-D Mapping using Information Theory , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[4]  Luc Jaulin,et al.  Localisation et suivi robustes d'un robot mobile grâce à l'analyse par intervalles Robust robot localization and tracking using interval analysis , 2000 .

[5]  Patrick Rives,et al.  Applying visual servoing techniques to control a mobile hand-eye system , 1995, Proceedings of 1995 IEEE International Conference on Robotics and Automation.

[6]  Ruzena Bajcsy,et al.  Control of visually guided behaviors , 1995 .

[7]  Eduardo Mario Nebot,et al.  Optimization of the simultaneous localization and map-building algorithm for real-time implementation , 2001, IEEE Trans. Robotics Autom..

[8]  Teresa A. Vidal-Calleja,et al.  On the Observability of Bearing-only SLAM , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[9]  Teresa A. Vidal-Calleja,et al.  Conditions for suboptimal filter stability in SLAM , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[10]  Javier Ibanez Guzman,et al.  On the Observability and Observability Analysis of SLAM , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[11]  Jake K. Aggarwal,et al.  Mobile robot self-location using model-image feature correspondence , 1996, IEEE Trans. Robotics Autom..

[12]  Andrew Zisserman,et al.  Detection and tracking of independent motion , 1995, Image and Vision Computing.

[13]  Juan Andrade-Cetto,et al.  The effects of partial observability when building fully correlated maps , 2005, IEEE Transactions on Robotics.

[14]  Jeffrey K. Uhlmann,et al.  Unscented filtering and nonlinear estimation , 2004, Proceedings of the IEEE.

[15]  James J. Little,et al.  Vision-based global localization and mapping for mobile robots , 2005, IEEE Transactions on Robotics.

[16]  Salah Sukkarieh,et al.  An information-theoretic approach to autonomous navigation and guidance of an uninhabited aerial vehicle in unknown environments , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[17]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[18]  M. S. Fadali,et al.  State Estimation , 2005 .

[19]  Man Hyung Lee,et al.  Localization of a mobile robot using images of a moving target , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[20]  Bruno Siciliano,et al.  Modeling and Control of Robot Manipulators , 1995 .

[21]  Juan D. Tardós,et al.  Data association in stochastic mapping using the joint compatibility test , 2001, IEEE Trans. Robotics Autom..

[22]  Udo Frese Treemap: An O(log n) algorithm for indoor simultaneous localization and mapping , 2006, Auton. Robots.

[23]  Roland Siegwart,et al.  Introduction to Autonomous Mobile Robots , 2004 .

[24]  Jeffrey K. Uhlmann,et al.  A counter example to the theory of simultaneous localization and map building , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[25]  Lindsay Kleeman,et al.  Mobile-Robot Map Building from an Advanced Sonar Array and Accurate Odometry , 1999, Int. J. Robotics Res..

[26]  Nobuyuki Kita,et al.  3D simultaneous localisation and map-building using active vision for a robot moving on undulating terrain , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[27]  Yoram Koren,et al.  The vector field histogram-fast obstacle avoidance for mobile robots , 1991, IEEE Trans. Robotics Autom..

[28]  Jean-Claude Latombe,et al.  Robot motion planning , 1970, The Kluwer international series in engineering and computer science.

[29]  David J. C. MacKay,et al.  Information-Based Objective Functions for Active Data Selection , 1992, Neural Computation.

[30]  Salah Sukkarieh,et al.  Active airborne localisation and exploration in unknown environments using inertial SLAM , 2006, 2006 IEEE Aerospace Conference.

[31]  Darius Burschka,et al.  Principles and practice of real-time visual tracking for navigation and mapping , 2004, International Workshop on Robot Sensing, 2004. ROSE 2004..

[32]  Peter I. Corke,et al.  A tutorial on visual servo control , 1996, IEEE Trans. Robotics Autom..

[33]  Liqiang Feng,et al.  Navigating Mobile Robots: Systems and Techniques , 1996 .

[34]  Matthew R. Walter,et al.  Sparse extended information filters: insights into sparsification , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[35]  H. Wang,et al.  A structure-from-motion algorithm for robot vehicle guidance , 1992, Proceedings of the Intelligent Vehicles `92 Symposium.

[36]  Danica Kragic,et al.  Robust Visual Servoing , 2003, Int. J. Robotics Res..

[37]  Robert Sim Stable Exploration for Bearings-only SLAM , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[38]  Michael Bosse,et al.  Mapping Partially Observable Features from Multiple Uncertain Vantage Points , 2002, Int. J. Robotics Res..

[39]  Wolfram Burgard,et al.  Autonomous exploration and mapping of abandoned mines , 2004, IEEE Robotics & Automation Magazine.

[40]  Sebastian Thrun,et al.  Robotic mapping: a survey , 2003 .

[41]  Keiji Nagatani,et al.  Topological simultaneous localization and mapping (SLAM): toward exact localization without explicit localization , 2001, IEEE Trans. Robotics Autom..

[42]  Klaus-Werner Jörg,et al.  Laserradar and sonar based world modeling and motion control for fast obstacle avoidance of the autonomous mobile robot MOBOT-IV , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[43]  Regis Hoffman,et al.  Terrain mapping for a walking planetary rover , 1994, IEEE Trans. Robotics Autom..

[44]  Ehud Rivlin,et al.  Computing the sensory uncertainty field of a vision-based localization sensor , 2001, IEEE Trans. Robotics Autom..

[45]  Eduardo Mario Nebot,et al.  Solving computational and memory requirements of feature-based simultaneous localization and mapping algorithms , 2003, IEEE Trans. Robotics Autom..

[46]  Weiping Li,et al.  Applied Nonlinear Control , 1991 .

[47]  Olivier Faugeras,et al.  Three-Dimensional Computer Vision , 1993 .

[48]  Gregory Dudek,et al.  Online control policy optimization for minimizing map uncertainty during exploration , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[49]  Hugh F. Durrant-Whyte,et al.  A new method for the nonlinear transformation of means and covariances in filters and estimators , 2000, IEEE Trans. Autom. Control..

[50]  Nicholas Roy,et al.  Global A-Optimal Robot Exploration in SLAM , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[51]  Roland Siegwart,et al.  Observability analysis for mobile robot localization , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[52]  Ehud Rivlin,et al.  Visual homing: Surfing on the epipoles , 1997, Block Island Workshop on Vision and Control.

[53]  R. W. Brockett,et al.  Asymptotic stability and feedback stabilization , 1982 .

[54]  J. A. Castellanos,et al.  Limits to the consistency of EKF-based SLAM , 2004 .

[55]  Warren E. Dixon,et al.  Adaptive tracking control of a wheeled mobile robot via an uncalibrated camera system , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[56]  Paul Newman,et al.  On the Structure and Solution of the Simultaneous Localisation and Map Building Problem , 1999 .

[57]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[58]  Hanumant Singh,et al.  Exactly Sparse Delayed-State Filters for View-Based SLAM , 2006, IEEE Transactions on Robotics.

[59]  Hans P. Moravec,et al.  High resolution maps from wide angle sonar , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[60]  J. Andrade-Cetto,et al.  Autonomous single camera exploration , 2006 .

[61]  David W. Murray,et al.  Real-time localization and mapping with wearable active vision , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[62]  Michel Devy,et al.  Undelayed initialization in bearing only SLAM , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[63]  SungJoon Kim Efficient simultaneous localization and mapping algorithms using submap networks , 2004 .

[64]  Alexei Makarenko,et al.  Information based adaptive robotic exploration , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[65]  Andrew J. Davison,et al.  Active search for real-time vision , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[66]  James P. Ostrowski,et al.  Visual servoing with dynamics: control of an unmanned blimp , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[67]  John J. Leonard,et al.  Consistent, Convergent, and Constant-Time SLAM , 2003, IJCAI.

[68]  I. Bar-Itzhack,et al.  Observability analysis of piece-wise constant systems. I. Theory , 1992 .

[69]  John J. Leonard,et al.  Adaptive Mobile Robot Navigation and Mapping , 1999, Int. J. Robotics Res..

[70]  Juan Andrade-Cetto,et al.  The effects of partial observability in SLAM , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[71]  Sebastian Thrun,et al.  Bayesian Landmark Learning for Mobile Robot Localization , 1998, Machine Learning.

[72]  Eduardo Nebot,et al.  Dense SLAM: The Unidirectional Information Flow (UIF) , 2004 .

[73]  Robert Sim Active Exploration Planning for SLAM using Extended Information Filters , 2004 .

[74]  Katsuhisa Furuta,et al.  State Variable Methods in Automatic Control , 1988 .

[75]  Ingemar J. Cox,et al.  Dynamic Map Building for an Autonomous Mobile Robot , 1992 .

[76]  Hans P. Moravec,et al.  The Stanford Cart and the CMU Rover , 1983, Proceedings of the IEEE.

[77]  James J. Little,et al.  Mobile Robot Localization and Mapping with Uncertainty using Scale-Invariant Visual Landmarks , 2002, Int. J. Robotics Res..

[78]  Andrew J. Davison,et al.  Real-time simultaneous localisation and mapping with a single camera , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[79]  Philippe Bonnifait,et al.  Design and experimental validation of an odometric and goniometric localization system for outdoor robot vehicles , 1998, IEEE Trans. Robotics Autom..

[80]  Olivier D. Faugeras,et al.  Some Properties of the E Matrix in Two-View Motion Estimation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[81]  Michael Bosse,et al.  Simultaneous Localization and Map Building in Large-Scale Cyclic Environments Using the Atlas Framework , 2004, Int. J. Robotics Res..

[82]  Gamini Dissanayake,et al.  Convergence analysis for extended Kalman filter based SLAM , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[83]  Sebastian Thrun,et al.  Learning Occupancy Grid Maps with Forward Sensor Models , 2003, Auton. Robots.

[84]  Camillo J. Taylor Building representations for the environment of a mobile robot from image data , 1992, Other Conferences.

[85]  Olivier D. Faugeras,et al.  Determination of Camera Location from 2-D to 3-D Line and Point Correspondences , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[86]  Wolfram Burgard,et al.  Probabilistic Algorithms and the Interactive Museum Tour-Guide Robot Minerva , 2000, Int. J. Robotics Res..

[87]  Eduardo Mario Nebot,et al.  Consistency of the EKF-SLAM Algorithm , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[88]  G.D. Hager,et al.  Toward domain-independent navigation: dynamic vision and control , 1998, Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171).

[89]  Simon J. Julier The stability of covariance inflation methods for SLAM , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[90]  W. Burgard,et al.  Markov Localization for Mobile Robots in Dynamic Environments , 1999, J. Artif. Intell. Res..

[91]  E. Malis,et al.  2 1/2 D Visual Servoing , 1999 .

[92]  Sebastian Thrun,et al.  Probabilistic Algorithms in Robotics , 2000, AI Mag..

[93]  David W. Murray,et al.  Simultaneous Localization and Map-Building Using Active Vision , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[94]  James P. Ostrowski,et al.  Visual motion planning for mobile robots , 2002, IEEE Trans. Robotics Autom..

[95]  A. Krener,et al.  Nonlinear controllability and observability , 1977 .

[96]  Dimitris P. Tsakiris,et al.  Applying Visual Servoing Techniques to Control Nonholonomic Mobile Robots , 1997 .

[97]  John J. Leonard,et al.  A Computationally Efficient Method for Large-Scale Concurrent Mapping and Localization , 2000 .

[98]  Yolanda González Cid,et al.  Real-time 3d SLAM with wide-angle vision , 2004 .

[99]  J. M. M. Montiel,et al.  The SPmap: a probabilistic framework for simultaneous localization and map building , 1999, IEEE Trans. Robotics Autom..

[100]  Antonio Bicchi,et al.  On the observability of mobile vehicles localization , 1998 .

[101]  Salah Sukkarieh,et al.  Improving the real-time efficiency of inertial SLAM and understanding its observability , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[102]  Alberto Sanfeliu,et al.  ESTIMATOR STABILITY ANALYSIS IN SLAM , 2004 .

[103]  Antonio Bicchi,et al.  Vision-based dynamic estimation and set-point stabilization of nonholonomic vehicles , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[104]  橋本 浩一 Visual servoing : real-time control of robot manipulators based on visual sensory feedback , 1993 .

[105]  Simon J. Julier,et al.  The spherical simplex unscented transformation , 2003, Proceedings of the 2003 American Control Conference, 2003..

[106]  R. Pfeifer,et al.  A mobile robot employing insect strategies for navigation , 2000, Robotics Auton. Syst..

[107]  Thia Kirubarajan,et al.  Estimation with Applications to Tracking and Navigation: Theory, Algorithms and Software , 2001 .

[108]  Michael Bosse,et al.  An Atlas framework for scalable mapping , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[109]  Robert E. Mahony,et al.  Visual servoing using linear features for under-actuated rigid body dynamics , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[110]  Juan Andrade-Cetto,et al.  Multirobot C-SLAM: Simultaneous localization, control, and mapping , 2005 .

[111]  Kai-Tai Song,et al.  Fast optical flow estimation and its application to real-time obstacle avoidance , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[112]  Claude Samson,et al.  Feedback control of a nonholonomic wheeled cart in Cartesian space , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[113]  Juan D. Tardós,et al.  Hierarchical SLAM: real-time accurate mapping of large environments , 2005, IEEE Transactions on Robotics.

[114]  Urbano Nunes,et al.  Path-following control of mobile robots in presence of uncertainties , 2005, IEEE Transactions on Robotics.

[115]  S. Sukkarieh,et al.  Autonomous airborne navigation in unknown terrain environments , 2004, IEEE Transactions on Aerospace and Electronic Systems.

[116]  Gamini Dissanayake,et al.  Multi-Step Look-Ahead Trajectory Planning in SLAM: Possibility and Necessity , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[117]  Peter Cheeseman,et al.  On the Representation and Estimation of Spatial Uncertainty , 1986 .

[118]  Darius Burschka,et al.  Vision-based control of mobile robots , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[119]  Darius Burschka,et al.  V-GPS - image-based control for 3D guidance systems , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[120]  Norihiko Adachi,et al.  Image-based visual adaptive tracking control of nonholonomic mobile robots , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[121]  Nikos A. Vlassis,et al.  Robot Planning in Partially Observable Continuous Domains , 2005, BNAIC.

[122]  Tim Bailey Constrained initialisation for bearing-only SLAM , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[123]  Teresa A. Vidal-Calleja,et al.  Unscented Transformation of Vehicle States in SLAM , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[124]  S. Sastry,et al.  Nonholonomic motion planning: steering using sinusoids , 1993, IEEE Trans. Autom. Control..

[125]  Hugh F. Durrant-Whyte,et al.  A Bayesian Algorithm for Simultaneous Localisation and Map Building , 2001, ISRR.

[126]  Klaus-Werner Jörg World modeling for an autonomous mobile robot using heterogenous sensor information , 1995, Robotics Auton. Syst..

[127]  Hugh F. Durrant-Whyte,et al.  Simultaneous Localization and Mapping with Sparse Extended Information Filters , 2004, Int. J. Robotics Res..

[128]  Vijay Kumar,et al.  Real-time vision-based control of a nonholonomic mobile robot , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[129]  Yasuhiro Masutani,et al.  Robot Motion Planning and Control for Moving Objects. , 1993 .

[130]  Wolfram Burgard,et al.  Information Gain-based Exploration Using Rao-Blackwellized Particle Filters , 2005, Robotics: Science and Systems.

[131]  Tom Duckett,et al.  Mobile robot self-localisation and measurement of performance in middle-scale environments , 1998, Robotics Auton. Syst..

[132]  H. Christensen,et al.  ROBUST SLAM , 2004 .

[133]  Frank P. Ferrie,et al.  Autonomous exploration: driven by uncertainty , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[134]  Hugh F. Durrant-Whyte,et al.  A solution to the simultaneous localization and map building (SLAM) problem , 2001, IEEE Trans. Robotics Autom..

[135]  Teresa A. Vidal-Calleja,et al.  Active control for single camera SLAM , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..