Estimating the deformability of elastic materials using optical flow and position-based dynamics

Knowledge of the physical properties of objects is essential in a wide range of robotic manipulation scenarios. A robot may not always be aware of such properties prior to interaction. If an object is incorrectly assumed to be rigid, it may exhibit unpredictable behavior when grasped. In this paper, we use vision based observation of the behavior of an object a robot is interacting with and use it as the basis for estimation of its elastic deformability. This is estimated in a local region around the interaction point using a physics simulator. We use optical flow to estimate the parameters of a position-based dynamics simulation using meshless shape matching (MSM). MSM has been widely used in computer graphics due to its computational efficiency, which is also important for closed-loop control in robotics. In a controlled experiment we demonstrate that our method can qualitatively estimate the physical properties of objects with different degrees of deformability.

[1]  Ronald Fedkiw,et al.  Finite volume methods for the simulation of skeletal muscle , 2003, SCA '03.

[2]  Wolfram Burgard,et al.  Tactile Sensing for Mobile Manipulation , 2011, IEEE Transactions on Robotics.

[3]  Doug L. James,et al.  FastLSM: fast lattice shape matching for robust real-time deformation , 2007, SIGGRAPH 2007.

[4]  Javier Sánchez Pérez,et al.  TV-L1 Optical Flow Estimation , 2013, Image Process. Line.

[5]  Hedvig Kjellström,et al.  Audio-visual classification and detection of human manipulation actions , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[6]  Kishore Reddy.  Action Recognition Using Particle Flow Fields , 2012 .

[7]  A OtaduyMiguel,et al.  A Survey on Position-Based Simulation Methods in Computer Graphics , 2014 .

[8]  Lieven Eeckhout,et al.  Deformable Surface 3D Reconstruction from Monocular Images , 2010 .

[9]  Advait Jain,et al.  Reaching in clutter with whole-arm tactile sensing , 2013, Int. J. Robotics Res..

[10]  Wolfram Burgard,et al.  Learning object deformation models for robot motion planning , 2014, Robotics Auton. Syst..

[11]  M. Teschner,et al.  Meshless deformations based on shape matching , 2005, SIGGRAPH 2005.

[12]  Murat Cenk Cavusoglu,et al.  Estimation of soft tissue mechanical parameters from robotic manipulation data , 2012, 2012 IEEE International Conference on Robotics and Automation.

[13]  Dieter Fox,et al.  RGB-D flow: Dense 3-D motion estimation using color and depth , 2013, 2013 IEEE International Conference on Robotics and Automation.

[14]  Masahiro Ohka,et al.  Low force control scheme for object hardness distinction in robot manipulation based on tactile sensing , 2008, 2008 IEEE International Conference on Robotics and Automation.

[15]  Dinesh K. Pai,et al.  ArtDefo: accurate real time deformable objects , 1999, SIGGRAPH.

[16]  Demetri Terzopoulos,et al.  A finite element model for 3D shape reconstruction and nonrigid motion tracking , 1993, 1993 (4th) International Conference on Computer Vision.

[17]  Bo Zhu,et al.  Simulation of organ deformation using boundary element method and meshless shape matching , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[18]  Niklas Bergström,et al.  Robust 3D tracking of unknown objects , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Ashutosh Saxena,et al.  Learning haptic representation for manipulating deformable food objects , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  Pascal Fua,et al.  Implicit Meshes for Effective Silhouette Handling , 2007, International Journal of Computer Vision.

[21]  Demetri Terzopoulos,et al.  Modelling and animating faces using scanned data , 1991, Comput. Animat. Virtual Worlds.

[22]  Advait Jain,et al.  Manipulation in Clutter with Whole-Arm Tactile Sensing , 2013, ArXiv.

[23]  Guangda Su,et al.  Multi-scale Method for Adaptive Mesh Editing Based on Rigidity Estimation , 2008, 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing.

[24]  Danica Kragic,et al.  SimTrack: A simulation-based framework for scalable real-time object pose detection and tracking , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[25]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[26]  A. D. Marshall,et al.  Learnt Real‐time Meshless Simulation , 2014, Comput. Graph. Forum.

[27]  Pieter Abbeel,et al.  Tracking deformable objects with point clouds , 2013, 2013 IEEE International Conference on Robotics and Automation.

[28]  Hans-Peter Seidel,et al.  MovieReshape: tracking and reshaping of humans in videos , 2010, ACM Trans. Graph..

[29]  Reinhard Koch,et al.  Simultaneous Estimation of Material Properties and Pose for Deformable Objects from Depth and Color Images , 2012, DAGM/OAGM Symposium.

[30]  Trevor Darrell,et al.  Using robotic exploratory procedures to learn the meaning of haptic adjectives , 2013, 2013 IEEE International Conference on Robotics and Automation.

[31]  Heinz Wörn,et al.  Robot Manipulation of Deformable Objects: Advanced Manufacturing , 2000 .

[32]  Wolfram Burgard,et al.  Learning the elasticity parameters of deformable objects with a manipulation robot , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[33]  Danica Kragic,et al.  What's in the container? Classifying object contents from vision and touch , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[34]  Miguel A. Otaduy,et al.  A Survey on Position‐Based Simulation Methods in Computer Graphics , 2014, Comput. Graph. Forum.

[35]  Jessica K. Hodgins,et al.  Capturing and animating skin deformation in human motion , 2006, SIGGRAPH '06.

[36]  Andrew Nealen,et al.  Physically Based Deformable Models in Computer Graphics , 2005, Eurographics.

[37]  Yin Yang,et al.  Haptic-enabled interactive rendering of deformable objects based on shape matching , 2013, 2013 IEEE International Symposium on Haptic Audio Visual Environments and Games (HAVE).

[38]  S. Matiacevich,et al.  Prediction of Mechanical Properties of Corn and Tortilla Chips by Using Computer Vision , 2012, Food and Bioprocess Technology.

[39]  Tae-Yong Kim,et al.  Unified particle physics for real-time applications , 2014, ACM Trans. Graph..