Hallucinating Humans for Learning Robotic Placement of Objects

While a significant body of work has been done on grasping objects, there is little prior work on placing and arranging objects in the environment. In this work, we consider placing multiple objects in complex placing areas, where neither the object nor the placing area may have been seen by the robot before. Specifically, the placements should not only be stable, but should also follow human usage preferences.We present learning and inference algorithms that consider these aspects in placing. In detail, given a set of 3D scenes containing objects, our method, based on Dirichlet process mixture models, samples human poses in each scene and learns how objects relate to those human poses. Then given a new room, our algorithm is able to select meaningful human poses and use them to determine where to place new objects.We evaluate our approach on a variety of scenes in simulation, as well as on robotic experiments.

[1]  Charles C. Kemp,et al.  Manipulation in Human Environments , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[2]  Pat Hanrahan,et al.  Context-based search for 3D models , 2010, ACM Trans. Graph..

[3]  Ashutosh Saxena,et al.  Reactive grasping using optical proximity sensors , 2009, 2009 IEEE International Conference on Robotics and Automation.

[4]  Michael Beetz,et al.  Equipping robot control programs with first-order probabilistic reasoning capabilities , 2009, 2009 IEEE International Conference on Robotics and Automation.

[5]  Bart Selman,et al.  Unstructured human activity detection from RGBD images , 2011, 2012 IEEE International Conference on Robotics and Automation.

[6]  Thorsten Joachims,et al.  Contextually guided semantic labeling and search for three-dimensional point clouds , 2013, Int. J. Robotics Res..

[7]  M. Escobar,et al.  Markov Chain Sampling Methods for Dirichlet Process Mixture Models , 2000 .

[8]  Oussama Khatib,et al.  Muscle force transmission to operational space accelerations during elite golf swings , 2012, 2012 IEEE International Conference on Robotics and Automation.

[9]  Yun Jiang,et al.  Learning Object Arrangements in 3D Scenes using Human Context , 2012, ICML.

[10]  Derek Hoiem,et al.  Recovering the spatial layout of cluttered rooms , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[11]  Esra Erdem,et al.  Housekeeping with Multiple Autonomous Robots : Knowledge Representation and Automated Reasoning for a Tightly Integrated Robot Control Architecture , 2011 .

[12]  Hiroshi Sugie,et al.  Placing objects with multiple mobile robots-mutual help using intention inference , 1995, Proceedings of 1995 IEEE International Conference on Robotics and Automation.

[13]  Sebastian Thrun,et al.  Video-based reconstruction of animatable human characters , 2010, ACM Trans. Graph..

[14]  Mark R. Cutkosky,et al.  Robotic grasping and fine manipulation , 1985 .

[15]  Daniel Huber,et al.  Using Context to Create Semantic 3D Models of Indoor Environments , 2010, BMVC.

[16]  Martial Hebert,et al.  Path planning with hallucinated worlds , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[17]  James M. Rehg,et al.  Perceiving clutter and surfaces for object placement in indoor environments , 2010, 2010 10th IEEE-RAS International Conference on Humanoid Robots.

[18]  Alberto Rodriguez,et al.  From caging to grasping , 2011, Int. J. Robotics Res..

[19]  Ashutosh Saxena,et al.  Cascaded Classification Models: Combining Models for Holistic Scene Understanding , 2008, NIPS.

[20]  Yee Whye Teh,et al.  Dirichlet Process , 2017, Encyclopedia of Machine Learning and Data Mining.

[21]  Vijay Kumar,et al.  Robotic grasping and contact: a review , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[22]  Oussama Khatib,et al.  Design and Control of a Bio-inspired Human-friendly Robot , 2010 .

[23]  Tsuhan Chen,et al.  Towards Holistic Scene Understanding: Feedback Enabled Cascaded Classification Models , 2010, NIPS.

[24]  Ashutosh Saxena,et al.  3-D Depth Reconstruction from a Single Still Image , 2007, International Journal of Computer Vision.

[25]  Steven M. LaValle,et al.  RRT-connect: An efficient approach to single-query path planning , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[26]  Thorsten Joachims,et al.  Contextually Guided Semantic Labeling and Search for 3D Point Clouds , 2011, ArXiv.

[27]  Pat Hanrahan,et al.  Characterizing structural relationships in scenes using graph kernels , 2011, SIGGRAPH 2011.

[28]  James J. Kuffner,et al.  OpenRAVE: A Planning Architecture for Autonomous Robotics , 2008 .

[29]  Ashutosh Saxena,et al.  Efficient grasping from RGBD images: Learning using a new rectangle representation , 2011, 2011 IEEE International Conference on Robotics and Automation.

[30]  Yun Jiang,et al.  Learning to place new objects in a scene , 2012, Int. J. Robotics Res..

[31]  Quoc V. Le,et al.  Learning to grasp objects with multiple contact points , 2010, 2010 IEEE International Conference on Robotics and Automation.

[32]  Ashutosh Saxena,et al.  Robotic Grasping of Novel Objects using Vision , 2008, Int. J. Robotics Res..

[33]  Tomás Lozano-Pérez,et al.  Task-level planning of pick-and-place robot motions , 1989, Computer.

[34]  TorralbaA.,et al.  Using the forest to see the trees , 2010 .

[35]  Ashutosh Saxena,et al.  Robotic Grasping of Novel Objects , 2006, NIPS.

[36]  Ruzena Bajcsy,et al.  Active Learning for Vision-Based Robot Grasping , 1996, Machine Learning.

[37]  Thorsten Joachims,et al.  Semantic Labeling of 3D Point Clouds for Indoor Scenes , 2011, NIPS.

[38]  Lawson L. S. Wong,et al.  Learning Grasp Strategies with Partial Shape Information , 2008, AAAI.

[39]  Antonio Torralba,et al.  Using the forest to see the trees: exploiting context for visual object detection and localization , 2010, CACM.

[40]  Ashutosh Saxena,et al.  Co-evolutionary predictors for kinematic pose inference from RGBD images , 2012, GECCO '12.

[41]  Michael Beetz,et al.  Parameterizing actions to have the appropriate effects , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[42]  Yun Jiang,et al.  Learning to place new objects , 2011, 2012 IEEE International Conference on Robotics and Automation.