Object Perception and Grasping in Open-Ended Domains

Nowadays service robots are leaving the structured and completely known environments and entering human-centric settings. For these robots, object perception and grasping are two challenging tasks due to the high demand for accurate and real-time responses. Although many problems have already been understood and solved successfully, many challenges still remain. Open-ended learning is one of these challenges waiting for many improvements. Cognitive science revealed that humans learn to recognize object categories and grasp affordances ceaselessly over time. This ability allows adapting to new environments by enhancing their knowledge from the accumulation of experiences and the conceptualization of new object categories. Inspired by this, an autonomous robot must have the ability to process visual information and conduct learning and recognition tasks in an open-ended fashion. In this context, "open-ended" implies that the set of object categories to be learned is not known in advance, and the training instances are extracted from online experiences of a robot, and become gradually available over time, rather than being completely available at the beginning of the learning process. In my research, I mainly focus on interactive open-ended learning approaches to recognize multiple objects and their grasp affordances concurrently. In particular, I try to address the following research questions: (i) What is the importance of open-ended learning for autonomous robots? (ii) How robots could learn incrementally from their own experiences as well as from interaction with humans? (iii) What are the limitations of Deep Learning approaches to be used in an open-ended manner? (iv) How to evaluate open-ended learning approaches and what are the right metrics to do so?

[1]  Xinyu Liu,et al.  Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics , 2017, Robotics: Science and Systems.

[2]  Luís Seabra Lopes,et al.  Interactive Open-Ended Object, Affordance and Grasp Learning for Robotic Manipulation , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[3]  Gi Hyun Lim,et al.  3D object perception and perceptual learning in the RACE project , 2016, Robotics Auton. Syst..

[4]  Gi Hyun Lim,et al.  A perceptual memory system for grounding semantic representations in intelligent service robots , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Gi Hyun Lim,et al.  Concurrent learning of visual codebooks and object categories in open-ended domains , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[6]  Hamidreza Kasaei,et al.  OrthographicNet: A Deep Learning Approach for 3D Object Recognition in Open-Ended Domains , 2019, ArXiv.

[7]  Dejan Pangercic,et al.  Robotic roommates making pancakes , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[8]  Luís Seabra Lopes,et al.  Hierarchical Object Representation for Open-Ended Object Category Learning and Recognition , 2016, NIPS.

[9]  Gi Hyun Lim,et al.  Towards lifelong assistive robotics: A tight coupling between object perception and manipulation , 2018, Neurocomputing.

[10]  Geoffrey A. Hollinger,et al.  HERB: a home exploring robotic butler , 2010, Auton. Robots.

[11]  Luís Seabra Lopes,et al.  GOOD: A global orthographic object descriptor for 3D object recognition and manipulation , 2016, Pattern Recognit. Lett..

[12]  Luís Seabra Lopes,et al.  Object Learning and Grasping Capabilities for Robotic Home Assistants , 2016, RoboCup.

[13]  Gi Hyun Lim,et al.  Interactive Open-Ended Learning for 3D Object Recognition: An Approach and Experiments , 2015, J. Intell. Robotic Syst..

[14]  Luís Seabra Lopes,et al.  Coping with Context Change in Open-Ended Object Recognition without Explicit Context Information , 2018, IROS.

[15]  Gi Hyun Lim,et al.  An Adaptive Object Perception System Based on Environment Exploration and Bayesian Learning , 2015, 2015 IEEE International Conference on Autonomous Robot Systems and Competitions.

[16]  Luís Seabra Lopes,et al.  Learning to grasp familiar objects using object view recognition and template matching , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[17]  Hamidreza Mohades Kasaei,et al.  OrthographicNet: A Deep Transfer Learning Approach for 3-D Object Recognition in Open-Ended Domains , 2019, IEEE/ASME Transactions on Mechatronics.

[18]  Luís Seabra Lopes,et al.  An orthographic descriptor for 3D object learning and recognition , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[19]  Sergey Levine,et al.  Learning Hand-Eye Coordination for Robotic Grasping with Large-Scale Data Collection , 2016, ISER.

[20]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Jörn Malzahn,et al.  WALK‐MAN: A High‐Performance Humanoid Platform for Realistic Environments , 2017, J. Field Robotics.

[22]  Gi Hyun Lim,et al.  Interactive teaching and experience extraction for learning about objects and robot activities , 2014, The 23rd IEEE International Symposium on Robot and Human Interactive Communication.

[23]  Gi Hyun Lim,et al.  Hierarchical Nearest Neighbor Graphs for Building Perceptual Hierarchies , 2015, ICONIP.

[24]  Tae-Kyun Kim,et al.  Perceiving, Learning, and Recognizing 3D Objects: An Approach to Cognitive Service Robots , 2018, AAAI.

[25]  Sergey Levine,et al.  Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[26]  Tae-Kyun Kim,et al.  Multi-view 6D Object Pose Estimation and Camera Motion Planning Using RGBD Images , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[27]  Tamim Asfour,et al.  Integrated Grasp and motion planning , 2010, 2010 IEEE International Conference on Robotics and Automation.

[28]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.