Interaction with virtual object using deformable hand

This study investigated the implementation of a hand model and contact simulation method for the purpose of improving the reality of object manipulation in a virtual environment. The study focused both on the hand tracking method that takes advantage of nails and also contact simulation using a deformable hand model. The manipulation of an object using a hand, is known to make more frequent use of the fingertips and palm. The proposed method seeks hand form that minimizes the position and orientation errors on those areas. Deformation of the soft tissue of the hand is considered to have an effect on both visual reality and the physical state of contact. In our implementation, the deformation was simulated by FEM and the friction of contact was introduced by the penalty method. In addition, a model that is based on metaballs (or blobs) was employed to represent the smooth surface of the object and to eliminate the problem that derives from polygon modeling. Through experimental implementation, it was proved that object manipulation such as pinching and grasping are possible and that the update rate of simulation can be approximately 50 Hz.

[1]  Bernd Fröhlich,et al.  A soft hand model for physically-based manipulation of virtual objects , 2011, 2011 IEEE Virtual Reality Conference.

[2]  V. Hayward,et al.  In vivo biomechanics of the fingerpad skin under local tangential traction. , 2007, Journal of biomechanics.

[3]  Maud Marchal,et al.  Aggregate Constraints for Virtual Manipulation with Soft Fingers , 2015, IEEE Transactions on Visualization and Computer Graphics.

[4]  Matei T. Ciocarlie,et al.  Soft Finger Model with Adaptive Contact Geometry for Grasping and Manipulation Tasks , 2007, Second Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC'07).

[5]  Wael Bachta,et al.  Analysis of the directions in which forces are applied on the hand during manual manipulation and exploration , 2015, 2015 IEEE World Haptics Conference (WHC).

[6]  Chen Qian,et al.  Realtime and Robust Hand Tracking from Depth , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Takuji Narumi,et al.  Controlling perceived stiffness of pinched objects using visual feedback of hand deformation , 2014, 2014 IEEE Haptics Symposium (HAPTICS).

[8]  John Kenneth Salisbury,et al.  Simulating human fingers: a soft finger proxy model and algorithm , 2004, 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2004. HAPTICS '04. Proceedings..

[9]  Vitor F. Pamplona,et al.  The Image-Based Data Glove , 2008 .

[10]  Michitaka Hirose,et al.  Dexterous object manipulation based on collision response , 2003, IEEE Virtual Reality, 2003. Proceedings..

[11]  Yuji Iwahori,et al.  A Study for Vision Based Data Glove Considering Hidden Fingertip with Self-Occlusion , 2012, 2012 13th ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing.

[12]  Leonard McMillan,et al.  Stable real-time deformations , 2002, SCA '02.

[13]  Christoph W. Borst,et al.  Realistic virtual grasping , 2005, IEEE Proceedings. VR 2005. Virtual Reality, 2005..

[14]  Kunihiro Chihara,et al.  Three-dimensional modeling of the human hand with motion constraints , 1999, Image Vis. Comput..

[15]  David Zeltzer,et al.  A survey of glove-based input , 1994, IEEE Computer Graphics and Applications.

[16]  Michael Garland,et al.  Efficient Sparse Matrix-Vector Multiplication on CUDA , 2008 .

[17]  Gabriel Zachmann,et al.  'Visual-fidelity' dataglove calibration , 2004 .

[18]  M. Bergamasco,et al.  Grasping and moving objects in virtual environments: a preliminary approach towards a realistic behaviour , 1994, Proceedings of 1994 3rd IEEE International Workshop on Robot and Human Communication.

[19]  Miguel A. Otaduy,et al.  Interactive simulation of a deformable hand for haptic rendering , 2011, 2011 IEEE World Haptics Conference.

[20]  N. Yamazaki,et al.  Relationship between structure of finger tissue and location of tactile receptors , 1997 .

[21]  C. D. Mote,et al.  Force response of the fingertip pulp to repeated compression--effects of loading rate, loading angle and anthropometry. , 1997, Journal of biomechanics.

[22]  Miguel A. Otaduy,et al.  Strain limiting for soft finger contact simulation , 2013, 2013 World Haptics Conference (WHC).

[23]  O. C. Zienkiewicz,et al.  The Finite Element Method: Its Basis and Fundamentals , 2005 .

[24]  R. Scopigno.  Contact Skinning , .

[25]  James F. Blinn,et al.  A Generalization of Algebraic Surface Drawing , 1982, TOGS.

[26]  Antonis A. Argyros,et al.  Efficient model-based 3D tracking of hand articulations using Kinect , 2011, BMVC.

[27]  Makiko Kouchi,et al.  Analysis of Skin Movements with Respect to Bone Motions using MR Images , 2003 .

[28]  Kazumi Kobayashi,et al.  Relationship between the Structure of Human Finger Tissue and the Location of Tactile Receptors , 1998 .

[29]  Jovan Popović,et al.  Real-time hand-tracking with a color glove , 2009, SIGGRAPH 2009.

[30]  Michael Neff,et al.  Data-driven glove calibration for hand motion capture , 2013, SCA '13.

[31]  Jérémie Allard,et al.  Implicit FEM Solver on GPU for Interactive Deformation Simulation , 2011 .

[32]  John Z. Wu,et al.  Three-dimensional finite element simulations of the mechanical response of the fingertip to static and dynamic compressions , 2006, Computer methods in biomechanics and biomedical engineering.

[33]  Rin-ichiro Taniguchi,et al.  Real-Time 3D Hand Shape Estimation Based on Inverse Kinematics and Physical Constraints , 2005, ICIAP.