Semantic web-mining and deep vision for lifelong object discovery

Autonomous robots that are to assist humans in their daily lives must recognize and understand the meaning of objects in their environment. However, the open nature of the world means robots must be able to learn and extend their knowledge about previously unknown objects on-line. In this work we investigate the problem of unknown object hypotheses generation, and employ a semantic web-mining framework along with deep-learning-based object detectors. This allows us to make use of both visual and semantic features in combined hypotheses generation. Experiments on data from mobile robots in real world application deployments show that this combination improves performance over the use of either method in isolation.

[1]  Nico Blodow,et al.  RoboSherlock: Unstructured information processing for robot perception , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[2]  Martha Palmer,et al.  Verb Semantics and Lexical Selection , 1994, ACL.

[3]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[4]  Moritz Tenorth,et al.  Learning organizational principles in human environments , 2012, 2012 IEEE International Conference on Robotics and Automation.

[5]  Markus Vincze,et al.  Autonomous Learning of Object Models on a Mobile Robot , 2017, IEEE Robotics and Automation Letters.

[6]  José Paulo Leal,et al.  Computing Semantic Relatedness using DBPedia , 2012, SLATE.

[7]  Simone Paolo Ponzetto,et al.  BabelNet: The automatic construction, evaluation and application of a wide-coverage multilingual semantic network , 2012, Artif. Intell..

[8]  Michael Isard,et al.  Lost in quantization: Improving particular object retrieval in large scale image databases , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Michael Beetz,et al.  PR2 looking at things — Ensemble learning for unstructured information processing with Markov logic networks , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[10]  Lutz Frommberger,et al.  Structural knowledge transfer by spatial abstraction for reinforcement learning agents , 2010, Adapt. Behav..

[11]  John J. Leonard,et al.  Toward lifelong object segmentation from change detection in dense RGB-D maps , 2013, 2013 European Conference on Mobile Robots.

[12]  Roberto Navigli,et al.  NASARI: a Novel Approach to a Semantically-Aware Representation of Items , 2015, NAACL.

[13]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[14]  Jürgen Schmidhuber,et al.  A Machine Learning Approach to Visual Perception of Forest Trails for Mobile Robots , 2016, IEEE Robotics and Automation Letters.

[15]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[16]  Moritz Tenorth,et al.  KnowRob: A knowledge processing infrastructure for cognition-enabled robots , 2013, Int. J. Robotics Res..

[17]  Daniele Nardi,et al.  Living with robots: Interactive environmental knowledge acquisition , 2016, Robotics Auton. Syst..

[18]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[19]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[20]  Elena Cabrio,et al.  Towards Lifelong Object Learning by Integrating Situated Robot Perception and Semantic Web Mining , 2016, ECAI.

[21]  Honglak Lee,et al.  Deep learning for detecting robotic grasps , 2013, Int. J. Robotics Res..

[22]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[23]  Marc Hanheide,et al.  Exploiting Probabilistic Knowledge under Uncertain Sensing for Efficient Robot Behaviour , 2011, IJCAI.

[24]  Rares Ambrus,et al.  Meta-rooms: Building and maintaining long term spatial models in a dynamic world , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.