Learning Predictive State Representation for in-hand manipulation

We study the use of Predictive State Representation (PSR) for modeling of an in-hand manipulation task through interaction with the environment. We extend the original PSR model to a new domain of in-hand manipulation and address the problem of partial observability by introducing new kernel-based features that integrate both actions and observations. The model is learned directly from haptic data and is used to plan series of actions that rotate the object in the hand to a specific configuration by pushing it against a table. Further, we analyze the model's belief states using additional visual data and enable planning of action sequences when the observations are ambiguous. We show that the learned representation is geometrically meaningful by embedding labeled action-observation traces. Suitability for planning is demonstrated by a post-grasp manipulation example that changes the object state to multiple specified target configurations.

[1]  Michael R. James,et al.  Predictive State Representations: A New Theory for Modeling Dynamical Systems , 2004, UAI.

[2]  Oliver Brock,et al.  State Representation Learning in Robotics: Using Prior Knowledge about Physical Interaction , 2014, Robotics: Science and Systems.

[3]  Nello Cristianini,et al.  Classification using String Kernels , 2000 .

[4]  Oliver Brock,et al.  Exploitation of environmental constraints in human and robotic grasping , 2015, Int. J. Robotics Res..

[5]  Joelle Pineau,et al.  Modelling Sparse Dynamical Systems with Compressed Predictive State Representations , 2013, ICML.

[6]  Michael R. James,et al.  Learning predictive state representations in dynamical systems without reset , 2005, ICML.

[7]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[8]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[9]  Satinder P. Singh,et al.  On discovery and learning of models with predictive representations of state for agents with continuous actions and observations , 2007, AAMAS '07.

[10]  Byron Boots,et al.  Learning predictive models of a depth camera & manipulator from raw execution traces , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[11]  Manuel G. Catalano,et al.  Adaptive synergies for the design and control of the Pisa/IIT SoftHand , 2014, Int. J. Robotics Res..

[12]  Oliver Brock,et al.  Exploitation of Environmental Constraints in Human and Robotic Grasping , 2013, ISRR.

[13]  Danica Kragic,et al.  A probabilistic framework for task-oriented grasp stability assessment , 2013, 2013 IEEE International Conference on Robotics and Automation.

[14]  Byron Boots,et al.  Hilbert Space Embeddings of Predictive State Representations , 2013, UAI.

[15]  Fernando Torres Medina,et al.  Multi-fingered robotic hand planner for object reconfiguration through a rolling contact evolution model , 2013, 2013 IEEE International Conference on Robotics and Automation.

[16]  Byron Boots,et al.  An Online Spectral Learning Algorithm for Partially Observable Nonlinear Dynamical Systems , 2011, AAAI.

[17]  Siddhartha S. Srinivasa,et al.  A data-driven statistical framework for post-grasp manipulation , 2014, Int. J. Robotics Res..

[18]  Sebastian Thrun,et al.  Learning low dimensional predictive representations , 2004, ICML.

[19]  Doina Precup,et al.  Point-Based Planning for Predictive State Representations , 2008, Canadian Conference on AI.

[20]  John J. Craig,et al.  Articulated hands: Force control and kinematic issues , 1981 .

[21]  Oliver Brock,et al.  A Novel Type of Compliant, Underactuated Robotic Hand for Dexterous Grasping , 2014, Robotics: Science and Systems.

[22]  John Shawe-Taylor,et al.  Syllables and other String Kernel Extensions , 2002, ICML.

[23]  Siddhartha S. Srinivasa,et al.  Extrinsic dexterity: In-hand manipulation with external forces , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[24]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[25]  Carla E. Brodley,et al.  Proceedings of the twenty-first international conference on Machine learning , 2004, International Conference on Machine Learning.

[26]  Leslie Pack Kaelbling,et al.  Robust grasping under object pose uncertainty , 2011, Auton. Robots.

[27]  Byron Boots,et al.  Closing the learning-planning loop with predictive state representations , 2009, Int. J. Robotics Res..

[28]  R. Bellman.  A Markovian Decision Process , 1957 .