A Serious Game for Learning Ultrasound-Guided Needle Placement Skills

Ultrasound-guided needle placement is a key step in a lot of radiological intervention procedures such as biopsy, local anesthesia, and fluid drainage. To help training future intervention radiologists, we develop a serious game to teach the skills involved. We introduce novel techniques for realistic simulation and integrate game elements for active and effective learning. This game is designed in the context of needle placement training based on the some essential characteristics of serious games. Training scenarios are interactively generated via a block-based construction scheme. A novel example-based texture synthesis technique is proposed to simulate corresponding ultrasound images. Game levels are defined based on the difficulties of the generated scenarios. Interactive recommendation of desirable insertion paths is provided during the training as an adaptation mechanism. We also develop a fast physics-based approach to reproduce the shadowing effect of needles in ultrasound images. Game elements such as time-attack tasks, hints, and performance evaluation tools are also integrated in our system. Extensive experiments are performed to validate its feasibility for training.

[1]  Jean-Louis Dillenseger,et al.  Fast simulation of ultrasound images from a CT volume , 2009, Comput. Biol. Medicine.

[2]  Yolanda A. Rankin,et al.  Re-Purposing a Recreational Video Game as a Serious Game for Second Language Acquisition , 2010 .

[3]  Clint A. Bowers,et al.  Serious Game Design and Development: Technologies for Training and Learning , 2010 .

[4]  R. Cobbold Foundations of Biomedical Ultrasound , 2006 .

[5]  Phillip Johnson,et al.  Serious Games for Immersive Cultural Training: Creating a Living World , 2009, IEEE Computer Graphics and Applications.

[6]  Derek R. Magee,et al.  An augmented reality simulator for ultrasound guided needle placement training , 2007, Medical & Biological Engineering & Computing.

[7]  P. Morse Vibration and Sound , 1949, Nature.

[8]  Patrice L. Weiss,et al.  TheraGame: A home based virtual reality rehabilitation system , 2006 .

[9]  Stacy Marsella,et al.  Serious Games for Language Learning: How Much Game, How Much AI? , 2005, AIED.

[10]  Marc O. Ernst,et al.  Visually Guided Haptic Search , 2010, IEEE Transactions on Haptics.

[11]  R. F. Wagner,et al.  Statistics of Speckle in Ultrasound B-Scans , 1983, IEEE Transactions on Sonics and Ultrasonics.

[12]  K. Boone,et al.  Effect of skin impedance on image quality and variability in electrical impedance tomography: a model study , 1996, Medical and Biological Engineering and Computing.

[13]  Dimitrios K. Iakovidis,et al.  Fuzzy Local Binary Patterns for Ultrasound Texture Characterization , 2008, ICIAR.

[14]  Claudia Adams,et al.  The use of computer games as an educational tool: identification of appropriate game types and game elements , 1999, Br. J. Educ. Technol..

[15]  Nassir Navab,et al.  Real-Time Simulation of Medical Ultrasound from CT Images , 2008, MICCAI.

[16]  Michael D. Proctor,et al.  Helicopter Flight Training Through Serious Aviation Gaming , 2007 .

[17]  Abdulmotaleb El-Saddik,et al.  Serious games , 2011, ACM Multimedia.

[18]  Sue Blackman,et al.  Serious games...and less! , 2005, COMG.

[19]  A. Basarab,et al.  Lateral RF image synthesis using a synthetic aperture imaging technique , 2008, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[20]  Musa Mailah,et al.  A PC-based driving simulator using virtual reality technology , 2004, VRCAI '04.

[21]  A. Bodenham,et al.  Visualisation of needle position using ultrasonography , 2006, Anaesthesia.

[22]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  A. Jemal,et al.  Cancer Statistics, 2008 , 2008, CA: a cancer journal for clinicians.

[24]  J. Arendt Paper presented at the 10th Nordic-Baltic Conference on Biomedical Imaging: Field: A Program for Simulating Ultrasound Systems , 1996 .

[25]  Tobias Isenberg,et al.  Depth-Dependent Halos: Illustrative Rendering of Dense Line Data , 2009, IEEE Transactions on Visualization and Computer Graphics.

[26]  Kup-Sze Choi,et al.  Learning Blood Management in Orthopedic Surgery through Gameplay , 2010, IEEE Computer Graphics and Applications.

[27]  J C Bamber,et al.  Ultrasonic B-scanning: a computer simulation , 1980, Physics in medicine and biology.

[28]  Roger Smith,et al.  The Long History of Gaming in Military Training , 2010 .

[29]  Dieter Schmalstieg,et al.  Crepuscular Rays for Tumor Accessibility Planning , 2011, IEEE Transactions on Visualization and Computer Graphics.

[30]  Josep Fuster,et al.  The Barcelona approach: Diagnosis, staging, and treatment of hepatocellular carcinoma , 2004, Liver transplantation : official publication of the American Association for the Study of Liver Diseases and the International Liver Transplantation Society.

[31]  Charles E McCulloch,et al.  Ultrasound visibility of needles used for regional nerve block: an in vitro study , 2004, Regional Anesthesia & Pain Medicine.

[32]  Henry Kelly,et al.  How Games and Simulations Can Help Meet America's Challenges in Science Mathematics and Technology Education , 2010 .

[33]  Fernando Bello,et al.  Developing An Immersive Ultrasound Guided Needle Puncture Simulator , 2009, MMVR.

[34]  Franck Vidal,et al.  Developing a needle guidance virtual environment with patient-specific data and force feedback , 2005 .

[35]  V. Vilgrain,et al.  Assessment of the benefits and risks of percutaneous biopsy before surgical resection of hepatocellular carcinoma. , 2001, Journal of hepatology.

[36]  Ralf Steinmetz,et al.  Seamless Learning in Serious Games - How to Improve Seamless Learning-content Integration in Serious Games , 2011, CSEDU.

[37]  Loris Nanni,et al.  Local binary patterns variants as texture descriptors for medical image analysis , 2010, Artif. Intell. Medicine.

[38]  Soon Ki Jung,et al.  Timed Automata-Based Rehabilitation Training Game Design for the Affected Lower Extremity of Hemiparetic Patient , 2008, Trans. Edutainment.

[39]  Septimiu E. Salcudean,et al.  Real-Time Image-Based B-Mode Ultrasound Image Simulation of Needles Using Tensor-Product Interpolation , 2011, IEEE Transactions on Medical Imaging.

[40]  Simon S. M. Ho,et al.  An ultrasound-guided organ biopsy simulation with 6DOF haptic feedback. , 2008, Medical image computing and computer-assisted intervention : MICCAI ... International Conference on Medical Image Computing and Computer-Assisted Intervention.

[41]  J Fitchew,et al.  War games. , 1983, Nursing mirror.

[42]  Thomas Grau,et al.  Ultrasound Characteristics of Needles for Regional Anesthesia , 2007, Regional Anesthesia & Pain Medicine.