Robust Topological Navigation via Convolutional Neural Network Feature and Sharpness Measure

Visual navigation for mobile robots has emerged in recent years. Among the various methods, topological navigation using visual information provides a scalable map representation for large-scale mapping and navigation. A topological map is essentially a graph with keyframes as its nodes and adjacency relations as its edges. Previous topological mapping uses local feature descriptors, such as scale-invariant feature transform or Speeded-Up Robust Features, to select keyframes in mapping, localization, and estimate relative pose. In practice, local features are not robust for severe motion blur or large illumination change. In this paper, we improve topological mapping to make it more efficient and robust. First, we use a convolutional neural network (CNN) feature as the holistic image representation. The CNN feature can be used to effectively retrieve keyframes that have similar appearance from a topological map, and it is robust to motion blur and illumination change. Thus, it improves the performance for place recognition and robot relocalization. Second, we use sharpness measure to select high-quality keyframes and avoid selecting blurry ones. Third, an efficient and robust non-rigid matching method, vector field consensus, is used for efficient geometric verification and to retrieve the most similar keyframe. The qualitative and quantitative experimental results demonstrate that our method is satisfactory.

[1]  Sim Heng Ong,et al.  A robust global and local mixture distance based non-rigid point set registration , 2015, Pattern Recognit..

[2]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[3]  Stergios I. Roumeliotis,et al.  Autonomous Flights Through Image-Defined Paths , 2015, ISRR.

[4]  Junjun Jiang,et al.  Non-Rigid Point Set Registration with Robust Transformation Estimation under Manifold Regularization , 2017, AAAI.

[5]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[6]  David Nistér,et al.  Scalable Recognition with a Vocabulary Tree , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[7]  J. M. M. Montiel,et al.  ORB-SLAM: A Versatile and Accurate Monocular SLAM System , 2015, IEEE Transactions on Robotics.

[8]  David S. Doermann,et al.  Sharpness estimation for document and scene images , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[9]  John J. Leonard,et al.  Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age , 2016, IEEE Transactions on Robotics.

[10]  Yansheng Li,et al.  Feature guided Gaussian mixture model with semi-supervised EM and local geometric constraint for retinal image registration , 2017, Inf. Sci..

[11]  Davide Scaramuzza,et al.  MAV urban localization from Google street view data , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[12]  Alan L. Yuille,et al.  Semi-Supervised Sparse Representation Based Classification for Face Recognition With Insufficient Labeled Samples , 2016, IEEE Transactions on Image Processing.

[13]  Ming Liu,et al.  Deep-learning in Mobile Robotics - from Perception to Control Systems: A Survey on Why and Why not , 2016, ArXiv.

[14]  Zhuowen Tu,et al.  Robust Point Matching via Vector Field Consensus , 2014, IEEE Transactions on Image Processing.

[15]  Jean-Arcady Meyer,et al.  Incremental vision-based topological SLAM , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[16]  Gang Wang,et al.  Robust Non-Rigid Point Set Registration Using Spatially Constrained Gaussian Fields , 2017, IEEE Transactions on Image Processing.

[17]  Zheng Wang,et al.  SRLSP: A Face Image Super-Resolution Algorithm Using Smooth Regression With Local Structure Prior , 2017, IEEE Transactions on Multimedia.

[18]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[19]  Zhuowen Tu,et al.  Regularized vector field learning with sparse approximation for mismatch removal , 2013, Pattern Recognit..

[20]  Junjun Jiang,et al.  Locality Preserving Matching , 2018, International Journal of Computer Vision.

[21]  Gang Wang,et al.  Context-Aware Gaussian Fields for Non-rigid Point Set Registration , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Dorian Gálvez-López,et al.  Bags of Binary Words for Fast Place Recognition in Image Sequences , 2012, IEEE Transactions on Robotics.

[23]  Alberto Ortiz,et al.  Hierarchical Place Recognition for Topological Mapping , 2017, IEEE Transactions on Robotics.

[24]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[25]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[26]  Jürgen Schmidhuber,et al.  A Machine Learning Approach to Visual Perception of Forest Trails for Mobile Robots , 2016, IEEE Robotics and Automation Letters.

[27]  Michael Milford,et al.  Convolutional Neural Network-based Place Recognition , 2014, ICRA 2014.

[28]  Andrew Zisserman,et al.  Return of the Devil in the Details: Delving Deep into Convolutional Nets , 2014, BMVC.

[29]  Stefano Carpin,et al.  Image-based mapping and navigation with heterogenous robots , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[30]  Jiayi Ma,et al.  Infrared and visible image fusion via gradient transfer and total variation minimization , 2016, Inf. Fusion.

[31]  Brian Cabral,et al.  Imaging vector fields using line integral convolution , 1993, SIGGRAPH.

[32]  Luc Van Gool,et al.  Omnidirectional Vision Based Topological Navigation , 2007, International Journal of Computer Vision.

[33]  Ruimin Hu,et al.  Face Super-Resolution via Multilayer Locality-Constrained Iterative Neighbor Embedding and Intermediate Dictionary Learning , 2014, IEEE Transactions on Image Processing.

[34]  Charles A. Micchelli,et al.  On Learning Vector-Valued Functions , 2005, Neural Computation.

[35]  Zhuowen Tu,et al.  Robust Estimation of Nonrigid Transformation for Point Set Registration , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Pierre Vandergheynst,et al.  FREAK: Fast Retina Keypoint , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[38]  Álvaro Sánchez Miralles,et al.  Topological simultaneous localization and mapping: a survey , 2013, Robotica.

[39]  Friedrich Fraundorfer,et al.  Topological mapping, localization and navigation using image collections , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[40]  Ruimin Hu,et al.  Noise Robust Face Hallucination via Locality-Constrained Representation , 2014, IEEE Transactions on Multimedia.

[41]  Sim Heng Ong,et al.  Remote Sensing Image Registration Using Multiple Image Features , 2017, Remote. Sens..

[42]  Ruimin Hu,et al.  CDMMA: Coupled discriminant multi-manifold analysis for matching low-resolution face images , 2016, Signal Process..

[43]  Niko Sünderhauf,et al.  On the performance of ConvNet features for place recognition , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[44]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[45]  Junjun Jiang,et al.  Noise Robust Face Image Super-Resolution Through Smooth Sparse Representation , 2017, IEEE Transactions on Cybernetics.