A Systematic Review of Real-Time Medical Simulations with Soft-Tissue Deformation: Computational Approaches, Interaction Devices, System Architectures, and Clinical Validations

Simulating deformations of soft tissues is a complex engineering task, and it is even more difficult when facing the constraint between computation speed and system accuracy. However, literature lacks of a holistic review of all necessary aspects (computational approaches, interaction devices, system architectures, and clinical validations) for developing an effective system of soft-tissue simulations. This paper summarizes and analyses recent achievements of resolving these issues to estimate general trends and weakness for future developments. A systematic review process was conducted using the PRISMA protocol with three reliable scientific search engines (ScienceDirect, PubMed, and IEEE). Fifty-five relevant papers were finally selected and included into the review process, and a quality assessment procedure was also performed on them. The computational approaches were categorized into mesh, meshfree, and hybrid approaches. The interaction devices concerned about combination between virtual surgical instruments and force-feedback devices, 3D scanners, biomechanical sensors, human interface devices, 3D viewers, and 2D/3D optical cameras. System architectures were analysed based on the concepts of system execution schemes and system frameworks. In particular, system execution schemes included distribution-based, multithread-based, and multimodel-based executions. System frameworks are grouped into the input and output interaction frameworks, the graphic interaction frameworks, the modelling frameworks, and the hybrid frameworks. Clinical validation procedures are ordered as three levels: geometrical validation, model behavior validation, and user acceptability/safety validation. The present review paper provides useful information to characterize how real-time medical simulation systems with soft-tissue deformations have been developed. By clearly analysing advantages and drawbacks in each system development aspect, this review can be used as a reference guideline for developing systems of soft-tissue simulations.

[1]  Thomas Sangild Sørensen,et al.  An Introduction to GPU Accelerated Surgical Simulation , 2006, ISBMS.

[2]  Ahmad H. Nasri,et al.  Mesh cutting during real-time physical simulation , 2011, Comput. Aided Des..

[3]  Cagatay Basdogan,et al.  Real-time visio-haptic interaction with static soft tissue models having geometric and material nonlinearity , 2010, Comput. Graph..

[4]  Hervé Delingette,et al.  Toward realistic soft-tissue modeling in medical simulation , 1998, Proc. IEEE.

[5]  Paul D. Gasson,et al.  Simulating plastic surgery: from human skin tensile tests, through hyperelastic finite element models to real-time haptics. , 2010, Progress in biophysics and molecular biology.

[6]  Takeshi Naemura,et al.  Internet communication using real-time facial expression analysis and synthesis , 2004, IEEE MultiMedia.

[7]  Stephane Cotin,et al.  A hybrid elastic model for real-time cutting, deformations, and force feedback for surgery training and simulation , 2000, The Visual Computer.

[8]  Fabio Lavagetto,et al.  Real-time MPEG-4 facial animation with 3D scalable meshes , 2002, Signal Process. Image Commun..

[9]  Luis Pastor,et al.  MSRS: A fast linear solver for the real-time simulation of deformable objects , 2008, Comput. Graph..

[10]  Mark A. Ganter,et al.  Real-time finite element modeling for surgery simulation: an application to virtual suturing , 2004, IEEE Transactions on Visualization and Computer Graphics.

[11]  Karol Miller,et al.  Real-Time Nonlinear Finite Element Computations on GPU - Application to Neurosurgical Simulation. , 2010, Computer methods in applied mechanics and engineering.

[12]  Icíar Alfaro,et al.  A comparison of implicit and explicit natural element methods in large strains problems: Application to soft biological tissues modeling , 2010 .

[13]  Christian Rössl,et al.  Laplacian surface editing , 2004, SGP '04.

[14]  Hervé Delingette,et al.  Fast porous visco-hyperelastic soft tissue model for surgery simulation: application to liver surgery. , 2010, Progress in biophysics and molecular biology.

[15]  Yongmin Zhong,et al.  A new ChainMail approach for real-time soft tissue simulation , 2016, Bioengineered.

[16]  Kun Zhou,et al.  Real-time facial animation on mobile devices , 2014, Graph. Model..

[17]  Mariano Alcañiz Raya,et al.  A new approach for the real-time simulation of tissue deformations in surgery simulation , 2001, Comput. Methods Programs Biomed..

[18]  Mohammad Eghtesad,et al.  Prediction of muscle activation for an eye movement with finite element modeling , 2017, Comput. Biol. Medicine.

[19]  Vincent Hayward,et al.  A PC-based system architecture for real-time finite element-based tool-specific surgical simulation , 2004, CARS.

[20]  Qing Li,et al.  Modelling of stress distribution and fracture in dental occlusal fissures , 2019, Scientific Reports.

[21]  Marie-Christine Ho Ba Tho,et al.  A Systematic Review of Continuum Modeling of Skeletal Muscles: Current Trends, Limitations, and Recommendations , 2018, Applied bionics and biomechanics.

[22]  Georgy Gimel'farb,et al.  A low cost framework for real-time marker based 3-D human expression modeling , 2017 .

[23]  Zhigang Deng,et al.  Marker Optimization for Facial Motion Acquisition and Deformation , 2013, IEEE Transactions on Visualization and Computer Graphics.

[24]  Stephane Cotin,et al.  EP4A: Software and Computer Based Simulator Research: Development and Outlook SOFA—An Open Source Framework for Medical Simulation , 2007, MMVR.

[25]  Elías Cueto,et al.  On the employ of meshless methods in biomechanics , 2005 .

[26]  Lixu Gu,et al.  Computerized Medical Imaging and Graphics a Hybrid Deformable Model for Real-time Surgical Simulation , 2022 .

[27]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. , 2010, International journal of surgery.

[28]  Shahin Sirouspour,et al.  GPU-based acceleration of computations in nonlinear finite element deformation analysis. , 2014, International journal for numerical methods in biomedical engineering.

[29]  Michael S Sacks,et al.  A Contemporary Look at Biomechanical Models of Myocardium. , 2019, Annual review of biomedical engineering.

[30]  Yohan Payan,et al.  Personalized modeling for real-time pressure ulcer prevention in sitting posture. , 2018, Journal of tissue viability.

[31]  A. Gefen,et al.  Real-time subject-specific monitoring of internal deformations and stresses in the soft tissues of the foot: a new approach in gait analysis. , 2006, Journal of biomechanics.

[32]  Gershon Elber,et al.  Real-time haptic incision simulation using FEM-based discontinuous free-form deformation , 2007, Comput. Aided Des..

[33]  Karol Miller,et al.  Patient-specific non-linear finite element modelling for predicting soft organ deformation in real-time: application to non-rigid neuroimage registration. , 2010, Progress in biophysics and molecular biology.

[34]  Fei Xu,et al.  The Effect of Endodontic Access Cavities on Fracture Resistance of First Maxillary Molar Using the Extended Finite Element Method , 2019, Journal of endodontics.

[35]  Philippe Pouletaut,et al.  IMAGE-BASED SKELETAL MUSCLE COORDINATION: CASE STUDY ON A SUBJECT SPECIFIC FACIAL MIMIC SIMULATION , 2018 .

[36]  K. Miller,et al.  Total Lagrangian explicit dynamics finite element algorithm for computing soft tissue deformation , 2006 .

[37]  Siamak Niroomandi,et al.  Accounting for large deformations in real-time simulations of soft tissues based on reduced-order models , 2012, Comput. Methods Programs Biomed..

[38]  BerkleyJeffrey,et al.  Real-Time Finite Element Modeling for Surgery Simulation , 2004 .

[39]  Christian Duriez,et al.  Real-time simulation of contact and cutting of heterogeneous soft-tissues , 2014, Medical Image Anal..

[40]  Karol Miller,et al.  An efficient hourglass control implementation for the uniform strain hexahedron using the Total Lagrangian formulation , 2007 .

[41]  Katsu Yamane,et al.  Musculoskeletal-see-through mirror: computational modeling and algorithm for whole-body muscle activity visualization in real time. , 2010, Progress in biophysics and molecular biology.

[42]  David S. Li,et al.  An integrated inverse model-experimental approach to determine soft tissue three-dimensional constitutive parameters: application to post-infarcted myocardium , 2017, Biomechanics and Modeling in Mechanobiology.

[43]  Suvranu De,et al.  Real time simulation of nonlinear tissue response in virtual surgery using the point collocation-based method of finite spheres , 2007 .

[44]  Antonio J. Serrano,et al.  A framework for modelling the biomechanical behaviour of the human liver during breathing in real time using machine learning , 2017, Expert Syst. Appl..

[45]  Jim X. Chen,et al.  Geodesic Distance-Based Realistic Facial Animation Using RBF Interpolation , 2012, Computing in Science & Engineering.

[46]  Karol Miller,et al.  Suite of finite element algorithms for accurate computation of soft tissue deformation for surgical simulation , 2009, Medical Image Anal..

[47]  S Niroomandi,et al.  Real‐time simulation of surgery by reduced‐order modeling and X‐FEM techniques , 2012, International journal for numerical methods in biomedical engineering.

[48]  Cagatay Basdogan,et al.  Real-Time Finite-Element Simulation of Linear Viscoelastic Tissue Behavior Based on Experimental Data , 2006, IEEE Computer Graphics and Applications.

[49]  Paul Suetens,et al.  Very fast soft tissue predictions with mass tensor model for maxillofacial surgery planning systems , 2005 .

[50]  Bing Zhou,et al.  Facial expression cloning with elastic and muscle models , 2014, J. Vis. Commun. Image Represent..

[51]  Andrew Nealen,et al.  Physically Based Deformable Models in Computer Graphics , 2006, Comput. Graph. Forum.

[52]  François Goulette,et al.  Fast computation of soft tissue deformations in real-time simulation with Hyper-Elastic Mass Links , 2015 .

[53]  Antonio J. Serrano,et al.  A finite element-based machine learning approach for modeling the mechanical behavior of the breast tissues under compression in real-time , 2017, Comput. Biol. Medicine.

[54]  Herve Delingette,et al.  Real-Time Elastic Deformations of Soft Tissues for Surgery Simulation , 1999, IEEE Trans. Vis. Comput. Graph..

[55]  Wei Sun,et al.  Recent development on computer aided tissue engineering - a review , 2002, Comput. Methods Programs Biomed..

[56]  Jean-Claude Latombe,et al.  Algorithmic tools for real-time microsurgery simulation , 2002, Medical Image Anal..

[57]  Yuri Bazilevs,et al.  A contact formulation based on a volumetric potential: Application to isogeometric simulations of atrioventricular valves. , 2018, Computer methods in applied mechanics and engineering.

[58]  Daniel Thalmann,et al.  Real time muscle deformations using mass-spring systems , 1998, Proceedings. Computer Graphics International (Cat. No.98EX149).

[59]  Ying Wang,et al.  Real time 3D simulation for nose surgery and automatic individual prosthesis design , 2011, Comput. Methods Programs Biomed..

[60]  Nadia Magnenat-Thalmann,et al.  Facial feature extraction for quick 3D face modeling , 2002, Signal Process. Image Commun..

[61]  Antonio Susín,et al.  A real-time dynamic 3D model of the human inguinal region for surgical education , 2007, Comput. Biol. Medicine.

[62]  Karol Miller,et al.  A three-dimensional nonlinear meshfree algorithm for simulating mechanical responses of soft tissue , 2014 .

[63]  Kenneth E. Barner,et al.  A Displacement Driven Real-Time Deformable Model For Haptic Surgery Simulation , 2006, 2006 14th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[64]  Hanif M Ladak,et al.  Virtual reality myringotomy simulation with real‐time deformation: Development and validity testing , 2012, The Laryngoscope.

[65]  Kumar Mithraratne,et al.  Modelling facial expressions: A framework for simulating nonlinear soft tissue deformations using embedded 3D muscles , 2013 .

[66]  Christian Duriez,et al.  GPU-based real-time soft tissue deformation with cutting and haptic feedback. , 2010, Progress in biophysics and molecular biology.

[67]  Ryo Kurazume,et al.  Navigation system with real-time finite element analysis for minimally invasive surgery , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[68]  Roger A. Sauer,et al.  Efficient isogeometric thin shell formulations for soft biological materials , 2016, Biomechanics and Modeling in Mechanobiology.

[69]  Guang-Zhong Yang,et al.  A machine learning approach for real-time modelling of tissue deformation in image-guided neurosurgery , 2017, Artif. Intell. Medicine.

[70]  Sébastien Ourselin,et al.  High-Speed Nonlinear Finite Element Analysis for Surgical Simulation Using Graphics Processing Units , 2008, IEEE Transactions on Medical Imaging.

[71]  Farzam Farahmand,et al.  Real-time simulation of the nonlinear visco-elastic deformations of soft tissues , 2010, International Journal of Computer Assisted Radiology and Surgery.

[72]  Jos Vander Sloten,et al.  Analyzing the potential of GPGPUs for real-time explicit finite element analysis of soft tissue deformation using CUDA , 2015 .

[73]  Chunquan Li,et al.  Real-time deformation of human soft tissues: A radial basis meshless 3D model based on Marquardt's algorithm , 2018, Comput. Methods Programs Biomed..