Estimating deformability of objects using meshless shape matching

Humans interact with deformable objects on a daily basis but this still represents a challenge for robots. To enable manipulation of and interaction with deformable objects, robots need to be able to extract and learn the deformability of objects both prior to and during the interaction. Physics-based models are commonly used to predict the physical properties of deformable objects and simulate their deformation accurately. The most popular simulation techniques are force-based models that need force measurements. In this paper, we explore the applicability of a geometry-based simulation method called meshless shape matching (MSM) for estimating the deformability of objects. The main advantages of MSM are its controllability and computational efficiency that make it popular in computer graphics to simulate complex interactions of multiple objects at the same time. Additionally, a useful feature of the MSM that differentiates it from other physics-based simulation is to be independent of force measurements that may not be available to a robotic framework lacking force/torque sensors. In this work, we design a method to estimate deformability based on certain properties, such as volume conservation. Using the finite element method (FEM) we create the ground truth deformability for various settings to evaluate our method. The experimental evaluation shows that our approach is able to accurately identify the deformability of test objects, supporting the value of MSM for robotic applications.

[1]  Stephane Cotin,et al.  EP4A: Software and Computer Based Simulator Research: Development and Outlook SOFA—An Open Source Framework for Medical Simulation , 2007, MMVR.

[2]  Gábor Székely,et al.  Simultaneous Topology and Stiffness Identification for Mass-Spring Models Based on FEM Reference Deformations , 2004, MICCAI.

[3]  Pieter Abbeel,et al.  Tracking deformable objects with point clouds , 2013, 2013 IEEE International Conference on Robotics and Automation.

[4]  Doug L. James,et al.  FastLSM: fast lattice shape matching for robust real-time deformation , 2007, SIGGRAPH 2007.

[5]  Jan Bender,et al.  Position-based simulation of continuous materials , 2014, Comput. Graph..

[6]  Peter Stone,et al.  Learning to Order Objects Using Haptic and Proprioceptive Exploratory Behaviors , 2016, IJCAI.

[7]  Reinhard Koch,et al.  Simultaneous Estimation of Material Properties and Pose for Deformable Objects from Depth and Color Images , 2012, DAGM/OAGM Symposium.

[8]  G. Pharr,et al.  Measurement of hardness and elastic modulus by instrumented indentation: Advances in understanding and refinements to methodology , 2004 .

[9]  Masahiro Ohka,et al.  Low force control scheme for object hardness distinction in robot manipulation based on tactile sensing , 2008, 2008 IEEE International Conference on Robotics and Automation.

[10]  Mårten Björkman,et al.  Learning to disambiguate object hypotheses through self-exploration , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[11]  Ashutosh Saxena,et al.  Learning haptic representation for manipulating deformable food objects , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[12]  Yunhui Liu,et al.  Automatic 3-D Manipulation of Soft Objects by Robotic Arms With an Adaptive Deformation Model , 2016, IEEE Transactions on Robotics.

[13]  Murat Cenk Cavusoglu,et al.  Estimation of soft tissue mechanical parameters from robotic manipulation data , 2012, 2012 IEEE International Conference on Robotics and Automation.

[14]  Ana-Maria Cretu,et al.  Soft Object Deformation Monitoring and Learning for Model-Based Robotic Hand Manipulation , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[15]  Oussama Khatib,et al.  Identifying physical properties of deformable objects by using particle filters , 2008, 2008 IEEE International Conference on Robotics and Automation.

[16]  Andrew Nealen,et al.  Physically Based Deformable Models in Computer Graphics , 2005, Eurographics.

[17]  Lieven Eeckhout,et al.  Deformable Surface 3D Reconstruction from Monocular Images , 2010 .

[18]  Advait Jain,et al.  Reaching in clutter with whole-arm tactile sensing , 2013, Int. J. Robotics Res..

[19]  Danica Kragic,et al.  What's in the container? Classifying object contents from vision and touch , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  Miguel A. Otaduy,et al.  A Survey on Position‐Based Simulation Methods in Computer Graphics , 2014, Comput. Graph. Forum.

[21]  Peter K. Allen,et al.  Integrating Vision and Touch for Object Recognition Tasks , 1988, Int. J. Robotics Res..

[22]  Connor Schenck,et al.  Detection and Tracking of Liquids with Fully Convolutional Networks , 2016, ArXiv.

[23]  Guangda Su,et al.  Multi-scale Method for Adaptive Mesh Editing Based on Rigidity Estimation , 2008, 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing.

[24]  W. Kabsch.  A solution for the best rotation to relate two sets of vectors , 1976 .

[25]  Tae-Yong Kim,et al.  Unified particle physics for real-time applications , 2014, ACM Trans. Graph..

[26]  Niklas Bergström,et al.  Robust tracking of unknown objects through adaptive size estimation and appearance learning , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[27]  Bo Zhu,et al.  Simulation of organ deformation using boundary element method and meshless shape matching , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[28]  Wojciech Matusik,et al.  Design and fabrication of materials with desired deformation behavior , 2010, ACM Trans. Graph..

[29]  Wolfram Burgard,et al.  Learning object deformation models for robot motion planning , 2014, Robotics Auton. Syst..

[30]  M. Teschner,et al.  Meshless deformations based on shape matching , 2005, SIGGRAPH 2005.

[31]  Danica Kragic,et al.  Estimating the deformability of elastic materials using optical flow and position-based dynamics , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).

[32]  Wolfram Burgard,et al.  Tactile Sensing for Mobile Manipulation , 2011, IEEE Transactions on Robotics.

[33]  Matthias Teschner,et al.  An implicit viscosity formulation for SPH fluids , 2015, ACM Trans. Graph..

[34]  Jeremy L. Wyatt,et al.  A Multimodal Model of Object Deformation Under Robotic Pushing , 2017, IEEE Transactions on Cognitive and Developmental Systems.

[35]  Niklas Bergström,et al.  Robust 3D tracking of unknown objects , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[36]  Trevor Darrell,et al.  Using robotic exploratory procedures to learn the meaning of haptic adjectives , 2013, 2013 IEEE International Conference on Robotics and Automation.

[37]  A. Nurnberger,et al.  A problem specific recurrent neural network for the description and simulation of dynamic spring models , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[38]  Yin Yang,et al.  Haptic-enabled interactive rendering of deformable objects based on shape matching , 2013, 2013 IEEE International Symposium on Haptic Audio Visual Environments and Games (HAVE).

[39]  Dmitry Berenson.  Autonomous and Human-Robot Collaborative Manipulation of 1 D and 2 D Deformable Objects Without Modeling Deformation , 2014 .