Building and sharing the cognition model of the environment with collaborative service robots

Aalto University, P.O. Box 11000, FI-00076 Aalto www.aalto.fi Author Sami Terho Name of the doctoral dissertation Building and sharing the cognition model of the environment with collaborative service robots Publisher School of Electrical Engineering Unit Department of Automation and Systems Technology Series Aalto University publication series DOCTORAL DISSERTATIONS 104/2011 Field of research Automation technology Manuscript submitted 24 August 2010 Manuscript revised 17 June 2011 Date of the defence 4 November 2011 Language English Monograph Article dissertation (summary + original articles) Abstract This research presents a shared cognition concept for building a cognition model collaboratively by robot and human. The proposed concept is based on an object-oriented approach that abstracts robot's perception, cognition, and knowledge to separate but interconnected elements. The shared cognition concept allows the human and robot share information related to objects in the environment for effective task execution and information exchange. The concept enables building a model using sensors, as well as inputting data from the user. User can utilize also robot’s sensor data for inputting information. This utilizes the robot’s perception of the environment in conjunction with human’s cognitive understanding.This research presents a shared cognition concept for building a cognition model collaboratively by robot and human. The proposed concept is based on an object-oriented approach that abstracts robot's perception, cognition, and knowledge to separate but interconnected elements. The shared cognition concept allows the human and robot share information related to objects in the environment for effective task execution and information exchange. The concept enables building a model using sensors, as well as inputting data from the user. User can utilize also robot’s sensor data for inputting information. This utilizes the robot’s perception of the environment in conjunction with human’s cognitive understanding. The proposed concept can be used for exchanging information on the objects, their classes, identities, appearances, locations, structures, and states. Representation of perceived entities is done through observed objects, robot's cognitive understanding of the objects is represented by real objects, and the knowledge of objects' classes by meta-objects. Observed objects are created with segmentation which can be autonomous or assisted by human. Real objects are used to transfer the actual object information between human and robot. Recognition of objects is done by matching real and observed objects. Uncertainties of recognition and localization are handled by probabilistic approach. The model enables learning from perceived information allowing the model to improve its performance as more data is gathered. The cognition model can be used for collaborative task execution. Several aspects of the model are validated with two different platforms, WorkPartner service robot and Avant machine based semi-autonomous robot. Based on the tests, the proposed model is an effective mechanism for collaboratively exchanging spatial information between a robot and a human.

[1]  Ravi Kiran Sarvadevabhatla,et al.  The memory game: Creating a human-robot interactive scenario for ASIMO , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  Andrew E. Johnson,et al.  Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Antonio Criminisi,et al.  TextonBoost: Joint Appearance, Shape and Context Modeling for Multi-class Object Recognition and Segmentation , 2006, ECCV.

[4]  Alexei A. Efros,et al.  Putting Objects in Perspective , 2006, CVPR.

[5]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[6]  John J. Craig Zhu,et al.  Introduction to robotics mechanics and control , 1991 .

[7]  Aarne Halme,et al.  Using signs for configuring work tasks of service robots , 2006, Ind. Robot.

[8]  Sami Ylonen.  Modularity in Service Robotics , 2008 .

[9]  Henrik I. Christensen,et al.  Bringing Together Human and Robotic Environment Representations - A Pilot Study , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Pascal Fua,et al.  A parallel stereo algorithm that produces dense depth maps and preserves image features , 1993, Machine Vision and Applications.

[11]  Victor Ng-Thow-Hing,et al.  Expanding task functionality in established humanoid robots , 2007, 2007 7th IEEE-RAS International Conference on Humanoid Robots.

[12]  V. Cherfaoui,et al.  Tracking objects using a laser scanner in driving situation based on modeling target shape , 2007, 2007 IEEE Intelligent Vehicles Symposium.

[13]  David G. Lowe,et al.  What and Where: 3D Object Recognition with Accurate Pose , 2006, Toward Category-Level Object Recognition.

[14]  Jussi Suomela,et al.  WorkPartner: Interactive Human-Like Service Robot for Outdoor Applications , 2003 .

[15]  Curtis W. Nielsen,et al.  Hiding the System from the User: Moving from Complex Mental Models to Elegant Metaphors , 2007, RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication.

[16]  Emanuele Trucco,et al.  Introductory techniques for 3-D computer vision , 1998 .

[17]  F. Tarsha-Kurdi,et al.  Hough-Transform and Extended RANSAC Algorithms for Automatic Detection of 3D Building Roof Planes from Lidar Data , 2007 .

[18]  Ravi Kiran Sarvadevabhatla,et al.  Cognitive map architecture , 2009, IEEE Robotics & Automation Magazine.

[19]  Fumio Kanehiro,et al.  Humanoid robot HRP-2 , 2008, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[20]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[21]  Nils J. Nilsson,et al.  A Formal Basis for the Heuristic Determination of Minimum Cost Paths , 1968, IEEE Trans. Syst. Sci. Cybern..

[22]  Y. Bar-Shalom.  Tracking and data association , 1988 .

[23]  Fumio Kanehiro,et al.  Cooperative works by a human and a humanoid robot , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[24]  S. Shankar Sastry,et al.  An Invitation to 3-D Vision , 2004 .

[25]  Wolfram Burgard,et al.  Supervised semantic labeling of places using information extracted from sensor data , 2007, Robotics Auton. Syst..

[26]  Yoshihiro Kawai,et al.  3D Object Recognition in Cluttered Environments by Segment-Based Stereo Vision , 2004, International Journal of Computer Vision.

[27]  Thierry Peynot,et al.  Error modeling and calibration of exteroceptive sensors for accurate mapping applications , 2010 .

[28]  P. Anandan,et al.  A computational framework and an algorithm for the measurement of visual motion , 1987, International Journal of Computer Vision.

[29]  Sebastian Thrun,et al.  Learning Metric-Topological Maps for Indoor Mobile Robot Navigation , 1998, Artif. Intell..

[30]  Shrihari Vasudevan,et al.  Spatial cognition for mobile robots , 2008 .

[31]  Marc Hanheide,et al.  Mixed-initiative in human augmented mapping , 2009, 2009 IEEE International Conference on Robotics and Automation.

[32]  J.-Y. Bouguet,et al.  Pyramidal implementation of the lucas kanade feature tracker , 1999 .

[33]  J. Canny.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Wolfram Burgard,et al.  Probabilistic Robotics (Intelligent Robotics and Autonomous Agents) , 2005 .

[35]  Mikko Heikkilä.  Configuration of skilled tasks for execution in multipurpose and collaborative service robots , 2009 .

[36]  Yoshihiro Kawai,et al.  Picking up an indicated object in a complex environment , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[37]  Sergiu Nedevschi,et al.  Road Surface and Obstacle Detection Based on Elevation Maps from Dense Stereo , 2007, 2007 IEEE Intelligent Transportation Systems Conference.

[38]  Kaoru Hirota,et al.  A Hu moment invariant as a shape circularity measure , 2010, Pattern Recognit..

[39]  Roland Siegwart,et al.  Cognitive maps for mobile robots - an object based approach , 2007, Robotics Auton. Syst..

[40]  Tobias Kaupp,et al.  Probabilistic Human-Robot Information Fusion , 2008 .

[41]  Rainer Lienhart,et al.  An extended set of Haar-like features for rapid object detection , 2002, Proceedings. International Conference on Image Processing.

[42]  Markus Ulrich,et al.  CAD-based recognition of 3D objects in monocular images , 2009, 2009 IEEE International Conference on Robotics and Automation.

[43]  Jean-Philippe Tarel,et al.  Real time obstacle detection in stereovision on non flat road geometry through "v-disparity" representation , 2002, Intelligent Vehicle Symposium, 2002. IEEE.

[44]  Alexei Makarenko,et al.  Shared environment representation for a human‐robot team performing information fusion , 2007, J. Field Robotics.

[45]  Emanuele Trucco,et al.  A compact algorithm for rectification of stereo pairs , 2000, Machine Vision and Applications.

[46]  C. Stachniss,et al.  Semantic Modeling of Places using Objects , 2008 .

[47]  Jussi Suomela,et al.  CREATING COMMON PRESENCE FOR A MULTIENTITY RESCUE TEAM , 2005 .

[48]  K. Yokoi,et al.  Operating Humanoid Robots in Human Environments , 2006 .

[49]  Matti Pietikäinen,et al.  Gray Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2000, ECCV.

[50]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[51]  Chin Seng Chua,et al.  Point Signatures: A New Representation for 3D Object Recognition , 1997, International Journal of Computer Vision.

[52]  George Vosselman,et al.  3D BUILDING MODEL RECONSTRUCTION FROM POINT CLOUDS AND GROUND PLANS , 2001 .

[53]  Anthony Stentz.  Optimal and Efficient Path Planning for Unknown and Dynamic Environments , 1993 .

[54]  Shuuji Kajita,et al.  Development of humanoid robot HRP-3P , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[55]  Jiri Matas,et al.  Robust wide-baseline stereo from maximally stable extremal regions , 2004, Image Vis. Comput..

[56]  Michael Brady,et al.  Ground plane estimation, error analysis and applications , 2002, Robotics Auton. Syst..

[57]  Ilkka Leppänen,et al.  Automatic locomotion mode control of wheel-legged robots , 2007 .

[58]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[59]  Joachim Hertzberg,et al.  Towards semantic maps for mobile robots , 2008, Robotics Auton. Syst..

[60]  Alessandro Saffiotti,et al.  Robot task planning using semantic maps , 2008, Robotics Auton. Syst..

[61]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[62]  Henrik I. Christensen,et al.  Topological Modelling for Human Augmented Mapping , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.