RoboSherlock: Unstructured information processing for robot perception

We present RoboSherlock, an open source software framework for implementing perception systems for robots performing human-scale everyday manipulation tasks. In RoboSherlock, perception and interpretation of realistic scenes is formulated as an unstructured information management (UIM) problem. The application of the UIM principle supports the implementation of perception systems that can answer task-relevant queries about objects in a scene, boost object recognition performance by combining the strengths of multiple perception algorithms, support knowledge-enabled reasoning about objects and enable automatic and knowledge-driven generation of processing pipelines. We demonstrate the potential of the proposed framework by three feasibility studies of systems for real-world scene perception that have been built on top of RoboSherlock.

[1]  Vincent Lepetit,et al.  Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenes , 2011, 2011 International Conference on Computer Vision.

[2]  Jennifer Chu-Carroll,et al.  Building Watson: An Overview of the DeepQA Project , 2010, AI Mag..

[3]  Gary R. Bradski,et al.  Recognition and Pose Estimation of Rigid Transparent Objects with a Kinect Sensor , 2012, Robotics: Science and Systems.

[4]  Stefan Holzer,et al.  Towards autonomous robotic butlers: Lessons learned with the PR2 , 2011, 2011 IEEE International Conference on Robotics and Automation.

[5]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[6]  Barbara Caputo,et al.  Multi-modal Semantic Place Classification , 2010, Int. J. Robotics Res..

[7]  Masayuki Inaba,et al.  Multi-cue 3D object recognition in knowledge-based vision-guided humanoid robot system , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[8]  Moritz Tenorth,et al.  A Knowledge Processing Service for Robots and Robotics/AI Researchers , 2015, ICRA 2015.

[9]  Gary R. Bradski,et al.  Fast 3D recognition and pose using the Viewpoint Feature Histogram , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[11]  Matthias Scheutz,et al.  ADE - Steps Towards a Distributed Development and Runtime Environment for Complex Robotic Agent Architectures , 2006 .

[12]  Moritz Tenorth,et al.  KNOWROB — knowledge processing for autonomous personal robots , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[13]  Dejan Pangercic,et al.  Semantic Object Maps for robotic housework - representation, acquisition and use , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  Óscar Martínez Mozos,et al.  Furniture Models Learned from the WWW , 2011, IEEE Robotics & Automation Magazine.

[15]  Siddhartha S. Srinivasa,et al.  The MOPED framework: Object recognition and pose estimation for manipulation , 2011, Int. J. Robotics Res..

[16]  Michael Beetz,et al.  PR2 looking at things — Ensemble learning for unstructured information processing with Markov logic networks , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[17]  Pierre Lison,et al.  Self-Understanding and Self-Extension: A Systems and Representational Approach , 2010, IEEE Transactions on Autonomous Mental Development.

[18]  Zoltan-Csaba Marton,et al.  Tutorial: Point Cloud Library: Three-Dimensional Object Recognition and 6 DOF Pose Estimation , 2012, IEEE Robotics & Automation Magazine.

[19]  Nico Blodow,et al.  Managing Belief States for Service Robots : Dynamic Scene Perception and Spatio-temporal Memory , 2014 .

[20]  Lucian Cosmin Goron,et al.  Segmenting Cylindrical and Box-like Objects in Cluttered 3D Scenes , 2012, Robotik 2012.

[21]  Gary R. Bradski,et al.  REIN - A fast, robust, scalable REcognition INfrastructure , 2011, 2011 IEEE International Conference on Robotics and Automation.

[22]  M. Vincze,et al.  BLORT-The Blocks World Robotic Vision Toolbox , 2010 .