Visual Semantic Navigation Based on Deep Learning for Indoor Mobile Robots

In order to improve the environmental perception ability of mobile robots during semantic navigation, a three-layer perception framework based on transfer learning is proposed, including a place recognition model, a rotation region recognition model, and a “side” recognition model. The first model is used to recognize different regions in rooms and corridors, the second one is used to determine where the robot should be rotated, and the third one is used to decide the walking side of corridors or aisles in the room. Furthermore, the “side” recognition model can also correct the motion of robots in real time, according to which accurate arrival to the specific target is guaranteed. Moreover, semantic navigation is accomplished using only one sensor (a camera). Several experiments are conducted in a real indoor environment, demonstrating the effectiveness and robustness of the proposed perception framework.

[1]  Mohamed Slim Masmoudi,et al.  Fuzzy Logic Based Control for Autonomous Mobile Robot Navigation , 2016, Comput. Intell. Neurosci..

[2]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[3]  Dejan Pangercic,et al.  Semantic Object Maps for robotic housework - representation, acquisition and use , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[4]  Patrick Rives,et al.  Semantic representation for navigation in large-scale environments , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[5]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Ali Farhadi,et al.  Target-driven visual navigation in indoor scenes using deep reinforcement learning , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Wolfram Burgard,et al.  Autonomous Robot Navigation in Highly Populated Pedestrian Zones , 2015, J. Field Robotics.

[8]  Guang Li,et al.  A Brief Review of Neural Networks Based Learning and Control and Their Applications for Robots , 2017, Complex..

[9]  Torsten Bertram,et al.  Visual Semantic Robot Navigation in Indoor Environments , 2014, ISR 2014.

[10]  Martin Buss,et al.  Route description interpretation on automatically labeled robot maps , 2013, 2013 IEEE International Conference on Robotics and Automation.

[11]  Vito Latora,et al.  The multiplex dependency structure of financial markets , 2016, Complex..

[12]  Patrick Rives,et al.  Fast hybrid relocation in large scale metric-topologic-semantic map , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[13]  Peter I. Corke,et al.  Visual Place Recognition: A Survey , 2016, IEEE Transactions on Robotics.

[14]  Michele Volpi,et al.  Dense Semantic Labeling of Subdecimeter Resolution Images With Convolutional Neural Networks , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Kai Oliver Arras,et al.  A Fast Random Walk Approach to Find Diverse Paths for Robot Navigation , 2017, IEEE Robotics and Automation Letters.

[16]  Il Hong Suh,et al.  Semantic mapping and navigation: A Bayesian approach , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[17]  Alessandro Saffiotti,et al.  Robot task planning using semantic maps , 2008, Robotics Auton. Syst..

[18]  Stefan Leutenegger,et al.  SemanticFusion: Dense 3D semantic mapping with convolutional neural networks , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Xiaoping Chen,et al.  Semantic mapping for object category and structural class , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  Pedro U. Lima,et al.  Efficient object search for mobile robots in dynamic environments: Semantic map as an input for the decision maker , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[21]  Taghi M. Khoshgoftaar,et al.  A survey of transfer learning , 2016, Journal of Big Data.

[22]  Dayal R. Parhi,et al.  Mobile robot navigation in unknown static environments using ANFIS controller , 2016 .

[23]  Lijun Zhao,et al.  Semantic region estimation of assistant robot for the elderly long-term operation in indoor environment , 2016, China Communications.

[24]  Rüdiger Dillmann,et al.  From Structure to Actions: Semantic Navigation Planning in Office Environments , 2011 .

[25]  Nick Barnes,et al.  Semantic labelling to aid navigation in prosthetic vision , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[26]  Jizhong Xiao,et al.  Visual semantic parameterization - To enhance blind user perception for indoor navigation , 2013, 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW).

[27]  Masayuki Inaba,et al.  Transformable semantic map based navigation using autonomous deep learning object segmentation , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[28]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Dieter Fox,et al.  DA-RNN: Semantic Mapping with Data Associated Recurrent Neural Networks , 2017, Robotics: Science and Systems.