A State-of-the-Art Review on Mapping and Localization of Mobile Robots Using Omnidirectional Vision Sensors

Nowadays, the field of mobile robotics is experiencing a quick evolution, and a variety of autonomous vehicles is available to solve different tasks. The advances in computer vision have led to a substantial increase in the use of cameras as the main sensors in mobile robots. They can be used as the only source of information or in combination with other sensors such as odometry or laser. Among vision systems, omnidirectional sensors stand out due to the richness of the information they provide the robot with, and an increasing number of works about them have been published over the last few years, leading to a wide variety of frameworks. In this review, some of the most important works are analysed. One of the key problems the scientific community is addressing currently is the improvement of the autonomy of mobile robots. To this end, building robust models of the environment and solving the localization and navigation problems are three important abilities that any mobile robot must have. Taking it into account, the review concentrates on these problems; how researchers have addressed them by means of omnidirectional vision; the main frameworks they have proposed; and how they have evolved in recent years.

[1]  Yachen Lin,et al.  Geometric Data Analysis: An Empirical Approach to Dimensionality Reduction and the Study of Patterns , 2002, Technometrics.

[2]  Darius Burschka,et al.  Advances in Computational Stereo , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Mei-Chen Yeh,et al.  Fast Human Detection Using a Cascade of Histograms of Oriented Gradients , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[4]  Denis Fernando Wolf,et al.  Outdoor Mapping Using Mobile Robots and Laser Range Finders , 2009, 2009 Electronics, Robotics and Automotive Mechanics Conference (CERMA).

[5]  Kostas Daniilidis,et al.  Direct 3D-rotation estimation from spherical images via a generalized shift theorem , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[6]  Shree K. Nayar,et al.  A theory of catadioptric image formation , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[7]  Pierre Vandergheynst,et al.  FREAK: Fast Retina Keypoint , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Masayuki Inaba,et al.  Visual navigation using omnidirectional view sequence , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[9]  Vincent Lepetit,et al.  BRIEF: Binary Robust Independent Elementary Features , 2010, ECCV.

[10]  Tom Duckett,et al.  Incremental Topological Mapping Using Omnidirectional Vision , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[11]  Soo-In Lee,et al.  Multi-robot Mapping Using Omnidirectional-Vision SLAM Based on Fisheye Images , 2014 .

[12]  João Pedro Barreto,et al.  Localization in indoor environments by querying omnidirectional visual maps using perspective images , 2012, 2012 IEEE International Conference on Robotics and Automation.

[13]  Nick Pears,et al.  Ground plane segmentation for mobile robot visual navigation , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[14]  Baoxin Li,et al.  Homography-based ground detection for a mobile robot platform using a single camera , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[15]  Roland Siegwart,et al.  A Flexible Technique for Accurate Omnidirectional Camera Calibration and Structure from Motion , 2006, Fourth IEEE International Conference on Computer Vision Systems (ICVS'06).

[16]  Alexander Zelinsky,et al.  Localisation using automatically selected landmarks from panoramic images , 2000 .

[17]  James J. Little,et al.  Vision-based global localization and mapping for mobile robots , 2005, IEEE Transactions on Robotics.

[18]  Jana Kosecka,et al.  Localization in Urban Environments Using a Panoramic Gist Descriptor , 2013, IEEE Transactions on Robotics.

[19]  Luis Payá,et al.  Appearance-based approach to hybrid metric-topological simultaneous localisation and mapping , 2014 .

[20]  Simone Gasparini,et al.  Camera Models and Fundamental Concepts Used in Geometric Computer Vision , 2011, Found. Trends Comput. Graph. Vis..

[21]  Óscar Martínez Mozos,et al.  A comparative evaluation of interest point detectors and local descriptors for visual SLAM , 2010, Machine Vision and Applications.

[22]  K. Hara,et al.  Navigation path scanning system for mobile robot by laser beam , 2008, 2008 SICE Annual Conference.

[23]  Emanuele Menegatti,et al.  Bayesian inference in the space of topological maps , 2006, IEEE Transactions on Robotics.

[24]  Shree K. Nayar,et al.  Generation of Perspective and Panoramic Video from Omnidirectional Video , 1997 .

[25]  Luis Payá,et al.  Using Omnidirectional Vision to Create a Model of the Environment: A Comparative Evaluation of Global-Appearance Descriptors , 2016, J. Sensors.

[26]  Roland Siegwart,et al.  A Robust Descriptor for Tracking Vertical Lines in Omnidirectional Images and Its Use in Mobile Robotics , 2009, Int. J. Robotics Res..

[27]  Timo Schairer,et al.  Visual mapping with uncertainty for correspondence-free localization using Gaussian process regression , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[28]  Koji Yoshida,et al.  An Omnidirectional Vision Sensor with Single Viewpoint and Constant Resolution , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[29]  Luis Payá,et al.  Appearance-based Dense Maps Creation - Comparison of Compression Techniques with Panoramic Images , 2009, ICINCO-RA.

[30]  Kostas Daniilidis,et al.  A Unifying Theory for Central Panoramic Systems and Practical Applications , 2000, ECCV.

[31]  Andrew Vardy,et al.  Visual homing in environments with anisotropic landmark distribution , 2007, Auton. Robots.

[32]  Alejandro Rituerto,et al.  Semantic labeling for indoor topological mapping using a wearable catadioptric system , 2014, Robotics Auton. Syst..

[33]  Hugh F. Durrant-Whyte,et al.  Mobile robot localization by tracking geometric beacons , 1991, IEEE Trans. Robotics Autom..

[34]  Joan Condell,et al.  Monocular Omnidirectional Vision based Robot Localisation and Mapping. , 2008 .

[35]  Norishige Chiba,et al.  Acquisition of spherical image by fish-eye conversion lens , 2004, IEEE Virtual Reality 2004.

[36]  Oliver Schlüter,et al.  Parsimonious loop-closure detection based on global image-descriptors of panoramic images , 2011, 2011 15th International Conference on Advanced Robotics (ICAR).

[37]  Daniel Cremers,et al.  Large-scale direct SLAM for omnidirectional cameras , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[38]  Georges Giralt,et al.  A Multi-level Planning and Navigation System for a Mobile Robot: A First Approach to HILARE , 1979, IJCAI.

[39]  Emanuele Menegatti,et al.  Image-based memory for robot navigation using properties of omnidirectional images , 2004, Robotics Auton. Syst..

[40]  Wen-Chung Chang,et al.  Vision-based robot navigation and map building using active laser projection , 2011, 2011 IEEE/SICE International Symposium on System Integration (SII).

[41]  Luis Payá,et al.  Estimation of Visual Maps with a Robot Network Equipped with Vision Sensors , 2010, Sensors.

[42]  Hong Zhang,et al.  Cylindrical panoramic image-based model for robot localization , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[43]  Roland Siegwart,et al.  BRISK: Binary Robust invariant scalable keypoints , 2011, 2011 International Conference on Computer Vision.

[44]  Valeri Kroumov,et al.  Localization and position correction for mobile robot using artificial visual landmarks , 2011, The 2011 International Conference on Advanced Mechatronic Systems.

[45]  Jean-Arcady Meyer,et al.  Visual topological SLAM and global localization , 2009, 2009 IEEE International Conference on Robotics and Automation.

[46]  Alejandro Rituerto,et al.  ´Indice de Contenidos A Comparison of omnidirectional and conventional monocular systems for visual SLAM , 2010 .

[47]  Emilio Garcia-Fidalgo,et al.  Vision-based topological mapping and localization methods: A survey , 2015, Robotics Auton. Syst..

[48]  Shigang Li,et al.  Full-View Spherical Image Camera , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[49]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[50]  Antonis A. Argyros,et al.  Robot Homing by Exploiting Panoramic Vision , 2005, Auton. Robots.

[51]  R. W. Wood,et al.  XXIII. Fish-eye views, and vision under water , 1906 .

[52]  Se-Young Oh,et al.  SLAM in Indoor Environments using Omni-directional Vertical and Horizontal Line Features , 2008, J. Intell. Robotic Syst..

[53]  David Valiente,et al.  A comparison of EKF and SGD applied to a view-based SLAM approach with omnidirectional images , 2014, Robotics Auton. Syst..

[54]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[55]  Valdir Grassi Junior,et al.  Development of an omnidirectional vision system , 2006 .

[56]  Tieniu Tan,et al.  Mobile robot self-localization based on global visual appearance features , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[57]  Timo Schairer,et al.  Learning to localize with Gaussian process regression on omnidirectional image data , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[58]  H. Friedrich,et al.  View-Based Robot Localization Using Spherical Harmonics: Concept and First Experimental Results , 2007, DAGM-Symposium.

[59]  Illah R. Nourbakhsh,et al.  Appearance-based place recognition for topological localization , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[60]  Yan Wang,et al.  The SLAM algorithm of mobile robot with omnidirectional vision based on EKF , 2012, 2012 IEEE International Conference on Information and Automation.

[61]  Renato Dourado Maia,et al.  Automatic plate detection using genetic algorithm , 2006 .

[62]  Derek Bradley,et al.  Image-based navigation in real environments using panoramas , 2005, IEEE International Workshop on Haptic Audio Visual Environments and their Applications.

[63]  Arnoud Visser,et al.  Creating a bird-eye view map using an omnidirectional camera , 2008 .

[64]  DaeEun Kim,et al.  Omni-directional image matching for homing navigation based on optical flow algorithm , 2012, 2012 12th International Conference on Control, Automation and Systems.

[65]  Avinash C. Kak,et al.  Vision for Mobile Robot Navigation: A Survey , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[66]  Luc Van Gool,et al.  Omnidirectional Vision Based Topological Navigation , 2007, International Journal of Computer Vision.

[67]  Shree K. Nayar,et al.  Cata-Fisheye Camera for Panoramic Imaging , 2008, 2008 IEEE Workshop on Applications of Computer Vision.

[68]  Huimin Lu,et al.  Robust place recognition based on omnidirectional vision and real-time local visual features for mobile robots , 2013, Adv. Robotics.

[69]  A. Leonardis,et al.  Robust localization using eigenspace of spinning-images , 2000, Proceedings IEEE Workshop on Omnidirectional Vision (Cat. No.PR00704).

[70]  Henrik I. Christensen,et al.  Localization and navigation of a mobile robot using natural point landmarks extracted from sonar data , 2000, Robotics Auton. Syst..

[71]  Timo Schairer,et al.  Increased accuracy orientation estimation from omnidirectional images using the spherical Fourier transform , 2009, 2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[72]  Timo Schairer,et al.  Application of particle filters to vision-based orientation estimation using harmonic analysis , 2010, 2010 IEEE International Conference on Robotics and Automation.

[73]  Kostas Daniilidis,et al.  Rotation recovery from spherical images without correspondences , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[74]  Antonio Torralba,et al.  Building the gist of a scene: the role of global image features in recognition. , 2006, Progress in brain research.

[75]  Yasushi Yagi,et al.  Panorama scene analysis with conic projection , 1990, EEE International Workshop on Intelligent Robots and Systems, Towards a New Frontier of Applications.

[76]  John J. Leonard,et al.  Robust Mapping and Localization in Indoor Environments Using Sonar Data , 2002, Int. J. Robotics Res..

[77]  Kurt Konolige,et al.  Navigation in hybrid metric-topological maps , 2011, 2011 IEEE International Conference on Robotics and Automation.

[78]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[79]  Li Zhang,et al.  Line segment based map building and localization using 2D laser rangefinder , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[80]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[81]  Antonio Maria Garcia Tommaselli,et al.  Calibration of a catadioptric omnidirectional vision system with conic mirror , 2016 .

[82]  Yasushi Yagi,et al.  Route representation for mobile robot navigation by omnidirectional route panorama Fourier transformation , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[83]  Koji Mori,et al.  A practical spherical mirror omnidirectional camera , 2005, International Workshop on Robotic Sensors: Robotic and Sensor Environments, 2005..

[84]  Gordon Wyeth,et al.  Persistent Navigation and Mapping using a Biologically Inspired SLAM System , 2010, Int. J. Robotics Res..

[85]  S. L. Bogner.  An introduction to panospheric imaging , 1995, 1995 IEEE International Conference on Systems, Man and Cybernetics. Intelligent Systems for the 21st Century.

[86]  Jana Kosecka,et al.  Qualitative image based localization in indoors environments , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[87]  Tarik Veli Mumcu,et al.  A Performance Analysis of Omnidirectional Vision Based Simultaneous Localization and Mapping , 2012, ICIC.

[88]  Alex Gurtner,et al.  Investigation of Fish-Eye Lenses for Small-UAV Aerial Photography , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[89]  Wolfram Burgard,et al.  Improving Simultaneous Mapping and Localization in 3D Using Global Constraints , 2005, AAAI.

[90]  James J. Little,et al.  Autonomous vision-based exploration and mapping using hybrid maps and Rao-Blackwellised particle filters , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[91]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[92]  Gaurav S. Sukhatme,et al.  Detection and Tracking of External Features in an Urban Environment Using an Autonomous Helicopter , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[93]  Eduardo L. L. Cabral,et al.  Omnidirectional stereo vision with a hyperbolic double lobed mirror , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[94]  Luis Payá,et al.  Towards relative altitude estimation in topological navigation tasks using the global appearance of visual information , 2014, 2014 International Conference on Computer Vision Theory and Applications (VISAPP).

[95]  Luis Payá,et al.  Position Estimation and Local Mapping Using Omnidirectional Images and Global Appearance Descriptors , 2015, Sensors.

[96]  Ben J. A. Kröse,et al.  Household robots look and learn: environment modeling and localization from an omnidirectional vision system , 2004, IEEE Robotics & Automation Magazine.

[97]  Roland Siegwart,et al.  Visual Homing From Scale With an Uncalibrated Omnidirectional Camera , 2013, IEEE Transactions on Robotics.

[98]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[99]  Luis Payá,et al.  Performance of Global-Appearance Descriptors in Map Building and Localization Using Omnidirectional Vision , 2014, Sensors.

[100]  Masayuki Inaba,et al.  Memory-Based Navigation using Omni-View Sequence , 1998 .

[101]  J. Gaspar,et al.  Omni-directional vision for robot navigation , 2000, Proceedings IEEE Workshop on Omnidirectional Vision (Cat. No.PR00704).

[102]  Boyan Bonev,et al.  Robot Navigation Behaviors based on Omnidirectional Vision and Information Theory , 2007 .

[103]  Emanuele Menegatti,et al.  Toward topological localization with spherical Fourier transform and uncalibrated camera , 2008 .

[104]  S. Nayar.  Omnidirectional Video Camera , 1997 .

[105]  Xianghua Ying,et al.  Catadioptric camera calibration using geometric invariants , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[106]  Nicholas Ayache,et al.  Trinocular Stereo Vision for Robotics , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[107]  Rudolf Mester,et al.  View-Based Robot Localization Using Illumination-Invariant Spherical Harmonics Descriptors , 2008, VISAPP.

[108]  Ana Cristina Murillo,et al.  SURF features for efficient robot localization with omnidirectional images , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[109]  José Santos-Victor,et al.  Vision-based navigation and environmental representations with an omnidirectional camera , 2000, IEEE Trans. Robotics Autom..

[110]  Ales Leonardis,et al.  Mobile robot localization using an incremental eigenspace model , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[111]  Kostas Daniilidis,et al.  Rotation estimation from spherical images , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[112]  Tianjiang Hu,et al.  BRISK based target localization for fixed-wing UAV's vision-based autonomous landing , 2015, 2015 IEEE International Conference on Information and Automation.

[113]  Masayuki Inaba,et al.  View-based approach to robot navigation , 2000, Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No.00CH37113).

[114]  Sumetee kesorn.  Visual Navigation for Mobile Robots: a Survey , 2012 .

[115]  Ales Leonardis,et al.  Unsupervised Learning of a Hierarchy of Topological Maps Using Omnidirectional Images , 2008, Int. J. Pattern Recognit. Artif. Intell..

[116]  Yasushi Yagi,et al.  Obstacle detection with omnidirectional image sensor HyperOmni Vision , 1995, Proceedings of 1995 IEEE International Conference on Robotics and Automation.

[117]  Sebastian Thrun,et al.  Robotic mapping: a survey , 2003 .

[118]  E. Trucco,et al.  Video Tracking: A Concise Survey , 2006, IEEE Journal of Oceanic Engineering.

[119]  Yunde Jia,et al.  Autonomous navigation of a miniature mobile robot using real-time trinocular stereo machine , 2003, IEEE International Conference on Robotics, Intelligent Systems and Signal Processing, 2003. Proceedings. 2003.

[120]  Emanuele Menegatti,et al.  Image-based Monte Carlo localisation with omnidirectional images , 2004, Robotics Auton. Syst..

[121]  Luis Payá,et al.  Map Building and Monte Carlo Localization Using Global Appearance of Omnidirectional Images , 2010, Sensors.

[122]  Chih-Hao Chen,et al.  Complete coverage motion control of a cleaning robot using infrared sensors , 2005, IEEE International Conference on Mechatronics, 2005. ICM '05..

[123]  Helder Araújo,et al.  Estimating parameters of noncentral catadioptric systems using bundle adjustment , 2009, Comput. Vis. Image Underst..

[124]  Liu Guang-liang,et al.  The Navigation of Mobile Robot Based on Stereo Vision , 2012, 2012 Fifth International Conference on Intelligent Computation Technology and Automation.

[125]  Kenji Imou,et al.  Localization of Agricultural Vehicle Using Landmarks Based on Omni-directional Vision , .

[126]  Hiroshi Murase,et al.  Visual learning and recognition of 3-d objects from appearance , 2005, International Journal of Computer Vision.

[127]  Chunhe Yu,et al.  A New 3D Map Reconstruction Based Mobile Robot Navigation , 2006, 2006 8th international Conference on Signal Processing.

[128]  Roland Siegwart,et al.  Scene recognition with omnidirectional vision for topological map using lightweight adaptive descriptors , 2009 .

[129]  Achim J. Lilienthal,et al.  SIFT, SURF & seasons: Appearance-based long-term localization in outdoor environments , 2010, Robotics Auton. Syst..

[130]  Gregory Dudek,et al.  Effective exploration strategies for the construction of visual maps , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[131]  정완균,et al.  Metric SLAM in Home Environment with Visual Objects and Sonar Features , 2006 .