Mobile Manipulation, Tool Use, and Intuitive Interaction for Cognitive Service Robot Cosero

Cognitive service robots that shall assist persons in need in performing their activities of daily living have recently received much attention in robotics research. Such robots require a vast set of control and perception capabilities to provide useful assistance through mobile manipulation and human-robot interaction. In this article, we present hardware design, perception, and control methods for our cognitive service robot Cosero. We complement autonomous capabilities with handheld teleoperation interfaces on three levels of autonomy. The robot demonstrated various advanced skills, including the use of tools. With our robot we participated in the annual international RoboCup@Home competitions, winning them three times in a row.

[1]  K. Dautenhahn,et al.  Care-O-bot 3 — Rationale for human-robot interaction design , 2008 .

[2]  H. Nguyen,et al.  ROS Commander : Flexible Behavior Creation for Home Robots , 2013 .

[3]  Stefan Holzer,et al.  Towards autonomous robotic butlers: Lessons learned with the PR2 , 2011, 2011 IEEE International Conference on Robotics and Automation.

[4]  Javier Ruiz-del-Solar,et al.  RoboCup@Home: Analysis and results of evolving competitions for domestic and service robots , 2015, Artif. Intell..

[5]  Jörg Stückler,et al.  Learning to interpret pointing gestures with a time-of-flight camera , 2011, 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[6]  Jörg Stückler,et al.  NimbRo Explorer: Semiautonomous Exploration and Mobile Manipulation in Rough Terrain , 2015, J. Field Robotics.

[7]  Florian Schmidt,et al.  Rollin' Justin - Mobile platform with variable base , 2009, 2009 IEEE International Conference on Robotics and Automation.

[8]  Dejan Pangercic,et al.  Robotic roommates making pancakes , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[9]  Tamim Asfour,et al.  ARMAR-III: An Integrated Humanoid Platform for Sensory-Motor Control , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[10]  Jörg Stückler,et al.  Increasing Flexibility of Mobile Manipulation and Intuitive Human-Robot Interaction in RoboCup@Home , 2013, RoboCup.

[11]  Ulrich Reiser,et al.  Let me Introduce Myself: I am Care-O-bot 4, a Gentleman Robot , 2015, MuC.

[12]  Jörg Stückler,et al.  Multi-resolution surfel maps for efficient dense 3D modeling and tracking , 2014, J. Vis. Commun. Image Represent..

[13]  Rüdiger Dillmann,et al.  Hardware and software architecture of the bimanual mobile manipulation robot HoLLiE and its actuated upper body , 2013, 2013 IEEE/ASME International Conference on Advanced Intelligent Mechatronics.

[14]  Jörg Stückler,et al.  Mobile Teleoperation Interfaces with Adjustable Autonomy for Personal Service Robots , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[15]  Jörg Stückler,et al.  Efficient deformable registration of multi-resolution surfel maps for object manipulation skill transfer , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[16]  Den Dolech,et al.  Tech United Eindhoven @Home 2015 Team Description Paper , 2015 .

[17]  Wolfram Burgard,et al.  Improved Techniques for Grid Mapping With Rao-Blackwellized Particle Filters , 2007, IEEE Transactions on Robotics.

[18]  Jörg Stückler,et al.  Cosero, Find My Keys! Object Localization and Retrieval Using Bluetooth Low Energy Tags , 2014, RoboCup.

[19]  Jörg Stückler,et al.  Adaptive tool-use strategies for anthropomorphic service robots , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[20]  Jun Li,et al.  Combining contour and shape primitives for object detection and pose estimation of prefabricated parts , 2013, 2013 IEEE International Conference on Image Processing.

[21]  Jörg Stückler,et al.  Efficient 3D object perception and grasp planning for mobile manipulation in domestic environments , 2013, Robotics Auton. Syst..

[22]  Erico Guizzo,et al.  The hard lessons of DARPA's robotics challenge [News] , 2015 .

[23]  Luca Iocchi,et al.  RoboCup@Home: Scientific Competition and Benchmarking for Domestic Service Robots , 2009 .

[24]  Jochen Kläß,et al.  Towards Robust Mobility, Flexible Object Manipulation, and Intuitive Multimodal Interaction for Domestic Service Robots , 2011, RoboCup.

[25]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[26]  Lydia E. Kavraki,et al.  Kinodynamic Motion Planning by Interior-Exterior Cell Exploration , 2008, WAFR.

[27]  Advait Jain,et al.  EL-E: an assistive mobile manipulator that autonomously fetches objects from flat surfaces , 2010, Auton. Robots.

[28]  Sven Behnke,et al.  Robot Competitions Ideal Benchmarks for Robotics Research , 2006 .

[29]  Sven Behnke,et al.  RGB-D object recognition and pose estimation based on pre-trained convolutional neural network features , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[30]  Kurt Konolige,et al.  Autonomous door opening and plugging in with a personal robot , 2010, 2010 IEEE International Conference on Robotics and Automation.

[31]  Jörg Stückler,et al.  Compliant Task-Space Control with Back-Drivable Servo Actuators , 2011, RoboCup.

[32]  Daniel Leidner,et al.  Object-centered hybrid reasoning for whole-body mobile manipulation , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[33]  Jochen Kläß,et al.  Efficient Mobile Robot Navigation using 3D Surfel Grid Maps , 2012, ROBOTIK.

[34]  Andriy Myronenko,et al.  Point Set Registration: Coherent Point Drift , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Shigeki Sugano,et al.  Design of human symbiotic robot TWENDY-ONE , 2009, 2009 IEEE International Conference on Robotics and Automation.

[36]  Sven Behnke,et al.  3D Body Pose Estimation Using an Adaptive Person Model for Articulated ICP , 2011, ICIRA.

[37]  Harold W. Kuhn,et al.  The Hungarian method for the assignment problem , 1955, 50 Years of Integer Programming.

[38]  Sven Behnke,et al.  Human-Like Interaction Skills for the Mobile Communication Robot Robotinho , 2013, Int. J. Soc. Robotics.

[39]  Jörg Stückler,et al.  Joint detection and pose tracking of multi-resolution surfel models in RGB-D , 2013, 2013 European Conference on Mobile Robots.

[40]  Xiaoping Chen,et al.  KeJia: The Intelligent Domestic Robot for RoboCup@Home 2015 , 2015 .

[41]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[42]  Jörg Stückler,et al.  NimbRo@Home: Winning Team of the RoboCup@Home Competition 2012 , 2012, RoboCup.

[43]  Jun Li,et al.  Mobile bin picking with an anthropomorphic service robot , 2013, 2013 IEEE International Conference on Robotics and Automation.

[44]  Dieter Fox,et al.  KLD-Sampling: Adaptive Particle Filters and Mobile Robot Localization , 2001, NIPS 2001.

[45]  Alexander Dietrich,et al.  Catching flying balls and preparing coffee: Humanoid Rollin'Justin performs dynamic and sensitive tasks , 2011, 2011 IEEE International Conference on Robotics and Automation.

[46]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[47]  Jörg Stückler,et al.  Adjustable autonomy for mobile teleoperation of personal service robots , 2012, 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication.

[48]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[49]  Geoffrey A. Hollinger,et al.  HERB: a home exploring robotic butler , 2010, Auton. Robots.

[50]  Sven Behnke,et al.  Humanoid Robots in Soccer: Robots Versus Humans in RoboCup 2050 , 2015, IEEE Robotics & Automation Magazine.

[51]  Jörg Stückler,et al.  Dynamaid, an Anthropomorphic Robot for Research on Domestic Service Applications , 2009, ECMR.

[52]  Sven Behnke,et al.  Intuitive bimanual telemanipulation under communication restrictions by immersive 3D visualization and motion tracking , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).

[53]  Sven Behnke,et al.  Controlling the gaze direction of a humanoid robot with redundant joints , 2008, RO-MAN 2008 - The 17th IEEE International Symposium on Robot and Human Interactive Communication.