How to add new knowledge to already trained deep learning models applied to semantic localization

The capacity of a robot to automatically adapt to new environments is crucial, especially in social robotics. Often, when these robots are deployed in home or office environments, they tend to fail because they lack the ability to adapt to new and continuously changing scenarios. In order to accomplish this task, robots must obtain new information from the environment, and then add it to their already learned knowledge. Deep learning techniques are often used to tackle this problem successfully. However, these approaches, complete retraining of the models, which is highly time-consuming. In this work, several strategies are tested to find the best way to include new knowledge in an already learned model in a deep learning pipeline, putting the spotlight on the time spent for this training. We tackle the localization problem in the long term with a deep learning approach and testing several retraining strategies. The results of the experiments are discussed and, finally, the best approach is deployed on a Pepper robot.

[1]  Cipriano Galindo,et al.  Multi-hierarchical semantic maps for mobile robotics , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  Margaret Lech,et al.  Object Recognition Using Deep Convolutional Features Transformed by a Recursive Network Structure , 2016, IEEE Access.

[3]  Sven Behnke,et al.  RGB-D object recognition and pose estimation based on pre-trained convolutional neural network features , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[4]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Edwin Olson,et al.  Learning semantic place labels from occupancy grids using CNNs , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[6]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[7]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[8]  Jose Miguel Puerta,et al.  Comparison between Bayesian network classifiers and SVMs for semantic localization , 2016, Expert Syst. Appl..

[9]  Miguel Cazorla,et al.  Scene classification based on semantic labeling , 2016, Adv. Robotics.

[10]  Wolfram Burgard,et al.  Multimodal deep learning for robust RGB-D object recognition , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[11]  Miguel Cazorla,et al.  Semantic localization in the PCL library , 2016, Robotics Auton. Syst..

[12]  Miguel Cazorla,et al.  ViDRILO: The Visual and Depth Robot Indoor Localization with Objects information dataset , 2015, Int. J. Robotics Res..

[13]  Rüdiger Dillmann,et al.  Learning Robot Behaviour and Skills Based on Human Demonstration and Advice: The Machine Learning Paradigm , 2000 .

[14]  José-Raúl Ruiz-Sarmiento,et al.  Building Multiversal Semantic Maps for Mobile Robot Operation , 2017, Knowl. Based Syst..

[15]  Bolei Zhou,et al.  Places: A 10 Million Image Database for Scene Recognition , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Peter I. Corke,et al.  Visual Place Recognition: A Survey , 2016, IEEE Transactions on Robotics.

[17]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[18]  Barbara Caputo,et al.  Multi-modal Semantic Place Classification , 2010, Int. J. Robotics Res..

[19]  Derek Hoiem,et al.  Learning without Forgetting , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Luis Rodríguez,et al.  On robot indoor scene classification based on descriptor quality and efficiency , 2017, Expert Syst. Appl..

[21]  Cristina Romero-Gonzlez,et al.  On robot indoor scene classification based on descriptor quality and efficiency , 2017 .

[22]  Rüdiger Dillmann,et al.  Teaching and learning of robot tasks via observation of human performance , 2004, Robotics Auton. Syst..