Approach to ease design criteria of a real-time model used in a VR training system considering constraints of human perception

Virtual reality (VR) systems have potential of contributing to the training of medical students in a variety of procedures. This thesis focuses on a design issue related to developing VR training systems for soft tissue (e.g., breast phantom) palpation. In such a VR system, it is paramount to provide a real-time model that simulates physical behavior of an actual breast phantom. However, it is difficult to design such a real-time model with high accuracy due to time and physical constraints. To mitigate this difficulty, I consider constraints of human perception which is insensitive to small discrepancies of objects during real-time interaction. Such consideration could aid to relax design criteria of the real-time model by achieving its accuracy at a certain degree while keeping human perception of object softness unchanged. Therefore, I take a two-step approach to determine visual and haptic (pertinent to force feedback) discrepancies tolerable for this human perception. In the first step, an evaluation method is developed to compute discrepancies of the real-time model for visual displacement and force feedback, compared to its finite element method counterpart featuring physical parameters of a breast phantom. The computation uses statistical analyses which like human perception are insensitive to small discrepancies of datasets. In the second step, two studies are performed to examine the constraints of human perception. The first study reexamined raw data from my MSc work to understand the effect of three popular alignments between a visual display and a haptic device on the human perception of object softness. This study serves to select an alignment producing the least perceptual illusion and physical workload for palpation. Using the evaluation method and the selected alignment, the second study investigates the effect of different discrepancies on the human perception of object softness. It is observed that this perception is insensitive to small discrepancies up to a threshold of 11.0% and 6.3% for visual displacement and force feedback,

[1]  David Swapp,et al.  Interaction with co-located haptic feedback in virtual reality , 2006, Virtual Reality.

[2]  Allison M. Okamura,et al.  The Touch Thimble: Providing Fingertip Contact Feedback During Point-Force Haptic Interaction , 2008, 2008 Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[3]  Yaoping Hu,et al.  Integration of the Senses of Vision and Touch in Perceiving Object Softness , 2007, 2007 Canadian Conference on Electrical and Computer Engineering.

[4]  Cagatay Basdogan,et al.  Haptics in virtual environments: taxonomy, research status, and challenges , 1997, Comput. Graph..

[5]  Laurent Étienne,et al.  Feeling bumps and holes without a haptic interface: the perception of pseudo-haptic textures , 2004, CHI.

[6]  M. A. Srinivassan.  The impact of visual information on the haptic perception of stiffness in virtual environments , 1996 .

[7]  Knut Drewing,et al.  Haptic, visual and visuo-haptic softness judgments for objects with deformable surfaces , 2009, World Haptics 2009 - Third Joint EuroHaptics conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[8]  Haruhisa Kawasaki,et al.  Medical training simulation for palpation of subsurface tumor using HIRO , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[9]  J M Bland,et al.  Statistical methods for assessing agreement between two methods of clinical measurement , 1986 .

[10]  R. Martyn Bracewell,et al.  Brain Mechanisms of Haptic Perception , 2008, The Sense of Touch and its Rendering.

[11]  A. Kappers,et al.  Haptic and visual perception of roughness. , 2007, Acta psychologica.

[12]  Carl Johan Lagerkvist.  Introductory Econometrics--Using Monte Carlo Simulation with Microsoft Excel , 2007 .

[13]  R. C. Oldfield.  The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[14]  K. T. Ramesh,et al.  Modelling of non-linear elastic tissues for surgical simulation , 2010, Computer methods in biomechanics and biomedical engineering.

[15]  Susan J. Lederman,et al.  Virtual peg-in-hole performance using a 6-DOF magnetic levitation haptic device: comparison with real forces and with visual guidance alone , 2002, Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002.

[16]  J. Shaffer.  Multiple Hypothesis Testing , 1995 .

[17]  Dinesh Manocha,et al.  Collision queries using oriented bounding boxes , 2000 .

[18]  Heinrich H. Bülthoff,et al.  Touch can change visual slant perception , 2000, Nature Neuroscience.

[19]  Susan J. Lederman,et al.  Sensing and Displaying Spatially Distributed Fingertip Forces in Haptic Interfaces for Teleoperator and Virtual Environment Systems , 1999, Presence: Teleoperators & Virtual Environments.

[20]  Fady T. Charbel,et al.  Virtual reality training in neurosurgery: Review of current status and future applications , 2011, Surgical neurology international.

[21]  R Bruyer,et al.  The Ponzo Illusion with Auditory Substitution of Vision in Sighted and Early-Blind Subjects , 2005, Perception.

[22]  Averill M. Law,et al.  Simulation Modeling and Analysis , 1982 .

[23]  R H LaMotte,et al.  Softness discrimination with a tool. , 2000, Journal of neurophysiology.

[24]  M A Srinivasan,et al.  Manual discrimination of compliance using active pinch grasp: The roles of force and work cues , 1995, Perception & psychophysics.

[25]  J. Stewart.  Calculus: Early Transcendentals , 1988 .

[26]  D. Bernstein.  Essentials of Psychology , 1998 .

[27]  Gershon Elber,et al.  Real-time haptic incision simulation using FEM-based discontinuous free-form deformation , 2007, Comput. Aided Des..

[28]  Matyka Maciej,et al.  Pressure Model of Soft Body Simulation , 2004, physics/0407003.

[29]  Jack McLaughlin,et al.  Haptic Workbench: a multisensory virtual environment , 1999, Electronic Imaging.

[30]  M. Heller,et al.  Intersensory conflict between vision and touch: The response modality dominates when precise, attention-riveting judgments are required , 1999, Perception & psychophysics.

[31]  George D. Stetten,et al.  Psychophysical evaluation of in-situ ultrasound visualization , 2005, IEEE Transactions on Visualization and Computer Graphics.

[32]  Jernej Barbic,et al.  Six-DoF Haptic Rendering of Contact Between Geometrically Complex Reduced Deformable Models , 2008, IEEE Transactions on Haptics.

[33]  Michael F Insana,et al.  Ultrasonic measurements of breast viscoelasticity. , 2007, Medical physics.

[34]  Stephane Cotin,et al.  Towards a Framework for Assessing Deformable Models in Medical Simulation , 2008, ISBMS.

[35]  Denis Laurendeau,et al.  Modelling liver tissue properties using a non-linear visco-elastic model for surgery simulation , 2005, Medical Image Anal..

[36]  Miriam Reiner,et al.  The role of haptics in immersive telecommunication environments , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[37]  R. Klatzky,et al.  Haptic Recognition of Static and Dynamic Expressions of Emotion in the Live Face , 2007, Psychological science.

[38]  John Dingliana,et al.  Collisions and perception , 2001, TOGS.

[39]  Antoine Widmer,et al.  The role of viewing angle in integrating the senses of vision and touch for perception of object softness , 2007, Canadian Journal of Electrical and Computer Engineering.

[40]  Mark A. Ganter,et al.  Real-time finite element modeling for surgery simulation: an application to virtual suturing , 2004, IEEE Transactions on Visualization and Computer Graphics.

[41]  I. Hunter,et al.  A perceptual analysis of stiffness , 2004, Experimental Brain Research.

[42]  Stephane Cotin,et al.  A hybrid elastic model for real-time cutting, deformations, and force feedback for surgery training and simulation , 2000, The Visual Computer.

[43]  Paul Suetens,et al.  Predicting soft tissue deformations for a maxillofacial surgery planning system: From computational strategies to a complete clinical validation , 2007, Medical Image Anal..

[44]  David S. Ebert,et al.  Haptic Identification of Stiffness and Force Magnitude , 2008, 2008 Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[45]  M. Serdar Celebi,et al.  Real-time deformation simulation of non-linear viscoelastic soft tissues , 2011, Simul..

[46]  Grigore C. Burdea,et al.  Virtual reality training simulation for palpation of subsurface tumors , 1997, Proceedings of IEEE 1997 Annual International Symposium on Virtual Reality.

[47]  Haruhisa Kawasaki,et al.  Medical palpation of deformable tissue using physics-based model for haptic interface robot (HIRO) , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[48]  Paul Suetens,et al.  Tetrahedral Mass Spring Model for Fast Soft Tissue Deformation , 2003, IS4TH.

[49]  Roberta L. Klatzky,et al.  Haptic Rendering and Psychophysical Evaluation of a Virtual Three-Dimensional Helical Spring , 2008, 2008 Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[50]  Hui Chen,et al.  Dynamic touch‐enabled virtual palpation , 2007, Comput. Animat. Virtual Worlds.

[51]  Christian Duriez,et al.  A multi-threaded approach for deformable/rigid contacts with haptic feedback , 2004, 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2004. HAPTICS '04. Proceedings..

[52]  Gino van den Bergen.  Efficient Collision Detection of Complex Deformable Models using AABB Trees , 1997, J. Graphics, GPU, & Game Tools.

[53]  Hui Chen,et al.  Body-Based Haptic Interaction Model for Touch-Enabled Virtual Environments , 2006, Presence: Teleoperators & Virtual Environments.

[54]  Martin Buss,et al.  Combination and Integration in the Perception of Visual-Haptic Compliance Information , 2010, IEEE Transactions on Haptics.

[55]  Hong Qin,et al.  Meshless methods for physics-based modeling and simulation of deformable models , 2009, Science in China Series F: Information Sciences.

[56]  Antoine Widmer,et al.  A viscoelastic model of a breast phantom for real-time palpation , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[57]  Antoine Widmer,et al.  Effects of the Alignment Between a Haptic Device and Visual Display on the Perception of Object Softness , 2010, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[58]  R. LaMotte,et al.  Magnitude estimation of softness , 2008, Experimental Brain Research.

[59]  Thomas M Greiner,et al.  Hand Anthropometry of U.S. Army Personnel , 1991 .

[60]  Martin Buss,et al.  Visual-Haptic Perception of Compliance: Fusion of Visual and Haptic Information , 2008, 2008 Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[61]  N. Mohsenin.  Physical properties of plant and animal materials , 1970 .

[62]  W. Lam,et al.  Factors affecting the palpability of breast lesion by self-examination. , 2008, Singapore medical journal.

[63]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[64]  S. Mangione.  Physical Diagnosis Secrets , 1999 .

[65]  Ming C. Lin,et al.  A modular haptic rendering algorithm for stable and transparent 6-DOF manipulation , 2006, IEEE Transactions on Robotics.

[66]  L. Kaufman,et al.  Handbook of perception and human performance , 1986 .

[67]  Gábor Székely,et al.  Identification of Spring Parameters for Deformable Object Simulation , 2007, IEEE Transactions on Visualization and Computer Graphics.

[68]  Aiguo Song,et al.  Discrimination and Memory Experiments on Haptic Perception of Softness , 2008, Perceptual and motor skills.

[69]  Stephane Cotin,et al.  Truth cube: Establishing physical standards for soft tissue simulation , 2003, Medical Image Anal..

[70]  Allison M. Okamura,et al.  Stiffness discrimination with visual and proprioceptive cues , 2009, World Haptics 2009 - Third Joint EuroHaptics conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[71]  Dennis Allerkamp,et al.  Virtual Reality Systems Modelling Haptic Two-Finger Contact with Deformable Physical Surfaces , 2007, 2007 International Conference on Cyberworlds (CW'07).

[72]  Thomas Kerwin,et al.  The role of multisensory feedback in haptic surface perception , 2003, 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2003. HAPTICS 2003. Proceedings..

[73]  Antoine Widmer,et al.  Statistical comparison between a real-time model and a FEM counterpart for visualization of breast phantom deformation during palpation , 2010, CCECE 2010.

[74]  Aiguo Song,et al.  Simulation of a mass-spring model for global deformation , 2009 .

[75]  Cagatay Basdogan,et al.  Real-Time Finite-Element Simulation of Linear Viscoelastic Tissue Behavior Based on Experimental Data , 2006, IEEE Computer Graphics and Applications.

[76]  Farzam Farahmand,et al.  Real-time simulation of the nonlinear visco-elastic deformations of soft tissues , 2010, International Journal of Computer Assisted Radiology and Surgery.

[77]  J. S. Lai,et al.  Creep and Relaxation of Nonlinear Viscoelastic Materials: With an Introduction to Linear Viscoelasticity , 2012 .

[78]  Abbas Samani,et al.  An iterative hyperelastic parameters reconstruction for breast cancer assessment , 2008, SPIE Medical Imaging.

[79]  Makoto Sato,et al.  Effect of Coupling Haptics and Stereopsis on Depth Perception in Virtual Environment , 2000 .

[80]  Colin Ware,et al.  Haptic state surface interactions , 2004, IEEE Computer Graphics and Applications.

[81]  George A. Gescheider,et al.  Psychophysics: The Fundamentals , 1997 .

[82]  Scott E. Maxwell,et al.  Designing Experiments and Analyzing Data: A Model Comparison Perspective , 1990 .

[83]  Abderrahmane Kheddar,et al.  Fast Continuous Collision Detection between Rigid Bodies , 2002, Comput. Graph. Forum.

[84]  D. Altman,et al.  Statistics notes: Measurement error , 1996 .