Augmented Reality for Digital Orthopedic Applications

Orthopedic diseases, either degenerative or osteoporotic, have become increasingly prevalent over the past two decades. Compression fracture, one of the most common forms of osteoporotic fractures, renders the patient completely disabled. With the increase in life expectancy, the demand for better healthcare to ensure a better life quality for older people has grown. Minimally invasive surgery (MIS) has presented itself as one of the major evolution in surgical techniques. As a greater benefit to the patients, it has reduced morbidity and recovery time after an intervention. This is a very significant advantage for orthopedic MIS. For example, C-arm images are required in spinal surgery. However, this introduces a major radiation hazard to both the patient and the surgeon. Another difficulty is the learning curve associated with this procedure. Currently, a novice surgeon requires many years of practice with an expert surgeon to perform interventions with a high degree of accuracy and safety. Thus, the purpose of the computer-aided surgery system is to develop an augmented reality guidance system for orthopedic surgery to facilitate in surgical procedures and through the process reduce the radiation exposure. Finally, the clinical trials of real-time augmented reality for spinal surgery and a proposed AR system depict in the last section of this chapter.

[1]  P. Kelly,et al.  Computer-assisted stereotaxic laser resection of intra-axial brain neoplasms. , 1986, Journal of neurosurgery.

[2]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[3]  B. Davies,et al.  Robotic surgery , 1993, IEEE Engineering in Medicine and Biology Magazine.

[4]  J. Dansereau,et al.  A points-to-surfaces matching technique for the application of augmented reality during spine surgery , 1995, Proceedings of 17th International Conference of the Engineering in Medicine and Biology Society.

[5]  D. Hill,et al.  Augmentation of reality using an operating microscope for otolaryngology and neurosurgical guidance. , 1995, Journal of image guided surgery.

[6]  E. Fishman,et al.  Three-dimensional volume rendering of spiral CT data: theory and method. , 1999, Radiographics : a review publication of the Radiological Society of North America, Inc.

[7]  A.J.N. van Breemen,et al.  Design and implementation of a room thermostat using an agent-based approach , 2001 .

[8]  Frank Sauer,et al.  An Augmented Reality Navigation System with a Single-Camera Tracker: System Design and Needle Biopsy Phantom Trial , 2002, MICCAI.

[9]  Wolfgang Birkfellner,et al.  A head-mounted operating binocular for augmented reality visualization in medicine - design and initial evaluation , 2002, IEEE Transactions on Medical Imaging.

[10]  Wolfgang Birkfellner,et al.  Computer-enhanced stereoscopic vision in a head-mounted operating binocular. , 2003, Physics in medicine and biology.

[11]  J. Kaspersen,et al.  Laparoscopic navigation pointer for three-dimensional image–guided surgery , 2004, Surgical Endoscopy And Other Interventional Techniques.

[12]  Bernhard Preim,et al.  Intraoperative augmented reality for minimally invasive liver interventions , 2003, SPIE Medical Imaging.

[13]  Nord Catharina The visible patient , 2003 .

[14]  Neil D. Glossop,et al.  Laser projection augmented reality system for computer-assisted surgery , 2003, CARS.

[15]  Ève Coste-Manière,et al.  Interactive Guidance by Image Overlay in Robot Assisted Coronary Artery Bypass , 2003, MICCAI.

[16]  S. Faraone,et al.  Stimulants and Attention-Deficit/Hyperactivity Disorder—Reply , 2004 .

[17]  J. Marescaux,et al.  Augmented-reality-assisted laparoscopic adrenalectomy. , 2004, JAMA.

[18]  O. Rueda,et al.  Computer-aided surgery based on auto-stereoscopic augmented reality , 2004 .

[19]  Ève Coste-Manière,et al.  Optimal Planning of Robotically Assisted Heart Surgery: First Results on the Transfer Precision in the Operating Room , 2004, Int. J. Robotics Res..

[20]  Daniel B. Russakoff,et al.  Intensity-based 2D-3D spine image registration incorporating a single fiducial marker. , 2005, Academic radiology.

[21]  Luc Soler,et al.  An augmented reality system to guide radio‐frequency tumour ablation , 2005, Comput. Animat. Virtual Worlds.

[22]  Heinz Wörn,et al.  Visualization of surgical 3D information with projector-based augmented reality. , 2005, Studies in health technology and informatics.

[23]  Daisuke Deguchi,et al.  Hybrid Bronchoscope Tracking Using a Magnetic Tracking Sensor and Image Registration , 2005, MICCAI.

[24]  Luc Soler,et al.  An augmented reality system to guide radio-frequency tumour ablation: Research Articles , 2005 .

[25]  J. Marescaux,et al.  Augmented Reality for Surgery and Interventional Therapy , 2005 .

[26]  Darius Burschka,et al.  Scale-invariant registration of monocular endoscopic images to CT-scans for sinus surgery , 2005, Medical Image Anal..

[27]  Luc Soler,et al.  A low cost and accurate guidance system for laparoscopic surgery: validation on an abdominal phantom , 2005, VRST '05.

[28]  T. Vogl,et al.  Accuracy of biopsy needle navigation using the Medarpa system—computed tomography reality superimposed on the site of intervention , 2005, European Radiology.

[29]  Philippe C. Cattin,et al.  Camera-marker alignment framework and comparison with hand-eye calibration for augmented reality applications , 2005, Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'05).

[30]  E. Lapresto,et al.  Image-guided endoscopy: description of technique and potential applications. , 2005, Neurosurgical focus.

[31]  T. Peters Image-guidance for surgical procedures , 2006, Physics in medicine and biology.

[32]  T. Vogl,et al.  Navigation-Based Needle Puncture of a Cadaver Using a Hybrid Tracking Navigational System , 2006, Investigative radiology.

[33]  David J. Hawkes,et al.  Cadaver validation of intensity-based ultrasound to CT registration , 2006, Medical Image Anal..

[34]  Nassir Navab,et al.  A New Registration/Visualization Paradigm for CT-Fluoroscopy Guided RF Liver Ablation , 2006, MICCAI.

[35]  Ekkehard Euler,et al.  Augmented reality visualization for thoracoscopic spine surgery , 2006, SPIE Medical Imaging.

[36]  Guang-Zhong Yang,et al.  Nonrigid 2-D/3-D Registration for Patient Specific Bronchoscopy Simulation With Statistical Shape Modeling: Phantom Validation , 2006, IEEE Transactions on Medical Imaging.

[37]  Guang-Zhong Yang,et al.  Simultaneous Stereoscope Localization and Soft-Tissue Mapping for Minimal Invasive Surgery , 2006, MICCAI.

[38]  J. Adler,et al.  Computer aided robotic radiosurgery , 2006, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[39]  Tina Kapur,et al.  Challenges in image-guided therapy system design , 2007, NeuroImage.

[40]  Pierre Graebling,et al.  Robust Structured Light Coding for 3D Reconstruction , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[41]  D. Jaffray,et al.  Review of image-guided radiation therapy , 2007, Expert review of anticancer therapy.

[42]  M. Feuerstein Augmented Reality in Laparoscopic Surgery , 2007 .

[43]  Makoto Hashizume,et al.  Recovery of respiratory motion and deformation of the liver using laparoscopic freehand 3D ultrasound system , 2007, Medical Image Anal..

[44]  R. Galloway,et al.  Laparoscopic surface scanning and subsurface targeting: implications for image-guided laparoscopic liver surgery. , 2007, Surgery.

[45]  Nassir Navab,et al.  Action- and Workflow-Driven Augmented Reality for Computer-Aided Medical Procedures , 2007, IEEE Computer Graphics and Applications.

[46]  J. Kettenbach,et al.  Rigid 2D/3D slice-to-volume registration and its application on fluoroscopic CT images. , 2006, Medical physics.

[47]  David J. Hawkes,et al.  MR Navigated Breast Surgery: Method and Initial Clinical Experience , 2008, MICCAI.

[48]  M. Figl,et al.  2D/3D registration of endoscopic ultrasound to CT volume data , 2008, Physics in medicine and biology.

[49]  Ken Masamune,et al.  Non-metal Slice Image Overlay Display System Used Inside the Open Type MRI , 2008, MIAR.

[50]  K. Cleary,et al.  Image-guided interventions : technology and applications , 2008 .

[51]  Peng Wen Medical Image Registration Based-on Points, Contour and Curves , 2008, 2008 International Conference on BioMedical Engineering and Informatics.

[52]  P. Romanelli,et al.  Spinal radiosurgery: technology and clinical outcomes , 2008, Neurosurgical Review.

[53]  K. Cleary,et al.  Image Guided Interventions. , 2020, Biomedizinische Technik. Biomedical engineering.

[54]  Nassir Navab,et al.  Intraoperative Laparoscope Augmentation for Port Placement and Resection Planning in Minimally Invasive Liver Resection , 2008, IEEE Transactions on Medical Imaging.

[55]  Guang-Zhong Yang,et al.  Real-time intra-operative 3D tissue deformation recovery , 2008, 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[56]  Vincenzo Ferrari,et al.  A 3-D Mixed-Reality System for Stereoscopic Visualization of Medical Dataset , 2009, IEEE Transactions on Biomedical Engineering.

[57]  Armin Schneider,et al.  Time-of-Flight 3-D Endoscopy , 2009, MICCAI.

[58]  G. Nollert,et al.  Planning a cardiovascular hybrid operating room: the technical point of view. , 2009, The heart surgery forum.

[59]  Michael Figl,et al.  Non-rigid Reconstruction of the Beating Heart Surface for Minimally Invasive Cardiac Surgery , 2009, MICCAI.

[60]  Nassir Navab,et al.  Magneto-Optical Tracking of Flexible Laparoscopic Ultrasound: Model-Based Online Detection and Correction of Magnetic Tracking Errors , 2009, IEEE Transactions on Medical Imaging.

[61]  J. Kettenbach,et al.  A comparative study on manual and automatic slice-to-volume registration of CT images , 2009, European Radiology.

[62]  L. Joskowicz,et al.  Surface-based facial scan registration in neuronavigation procedures: a clinical study. , 2009, Journal of neurosurgery.

[63]  Russell H. Taylor,et al.  Augmented reality during robot-assisted laparoscopic partial nephrectomy: toward real-time 3D-CT to stereoscopic video registration. , 2009, Urology.

[64]  Hongen Liao,et al.  3-D Augmented Reality for MRI-Guided Surgery Using Integral Videography Autostereoscopic Image Overlay , 2010, IEEE Transactions on Biomedical Engineering.

[65]  Soon-Yong Park,et al.  Active Calibration of Camera-Projector Systems Based on Planar Homography , 2010, 2010 20th International Conference on Pattern Recognition.

[66]  Guang-Zhong Yang,et al.  Motion Compensated SLAM for Image Guided Surgery , 2010, MICCAI.

[67]  Guang-Zhong Yang,et al.  Real-Time Stereo Reconstruction in Robotically Assisted Minimally Invasive Surgery , 2010, MICCAI.

[68]  G. May Image-Guided Interventions , 2010 .

[69]  Pierre Graebling,et al.  Real-time structured light patterns coding with subperfect submaps , 2010, Photonics Europe.

[70]  Luc Soler,et al.  Bulk modulus and volume variation measurement of the liver and the kidneys in vivo using abdominal kinetics during free breathing , 2010, Comput. Methods Programs Biomed..

[71]  Jiann-Der Lee,et al.  Fast-MICP for frameless image-guided surgery. , 2010, Medical physics.

[72]  Michael Figl,et al.  Image guidance for robotic minimally invasive coronary artery bypass , 2010, Comput. Medical Imaging Graph..

[73]  Bernhard Preim,et al.  Improved Navigated Spine Surgery Utilizing Augmented Reality Visualization , 2011, Bildverarbeitung für die Medizin.

[74]  Lena Maier-Hein,et al.  Time-of-flight camera technique for augmented reality in computer-assisted interventions , 2011, Medical Imaging.

[75]  J. Marescaux,et al.  Augmented reality in laparoscopic surgical oncology. , 2011, Surgical oncology.

[76]  Graeme P. Penney,et al.  An Image-Guided Surgery System to Aid Endovascular Treatment of Complex Aortic Aneurysms: Description and Initial Clinical Experience , 2011, IPCAI.

[77]  Adrien Bartoli,et al.  Tracking by Detection for Interactive Image Augmentation in Laparoscopy , 2012, WBIR.

[78]  Huei-Yung Lin,et al.  Geometric constraints for robot navigation using omnidirectional camera , 2012, 2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[79]  Joonwhoan Lee,et al.  A Lighting Insensitive Face Detection Method on Color Images , 2012, 2012 Spring Congress on Engineering and Technology.

[80]  Bostjan Likar,et al.  A review of 3D/2D registration methods for image-guided interventions , 2012, Medical Image Anal..

[81]  Sebastian Bodenstedt,et al.  Dense GPU-enhanced surface reconstruction from stereo endoscopic images for intraoperative registration. , 2012, Medical physics.

[82]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[83]  Thomas Hartkens,et al.  The Hybrid Operating Room , 2012 .

[84]  Michael Figl,et al.  Reconstruction of a 3D surface from video that is robust to missing data and outliers: Application to minimally invasive surgery using stereo and mono endoscopes , 2012, Medical Image Anal..

[85]  Adrien Bartoli,et al.  Template-Based Conformal Shape-from-Motion-and-Shading for Laparoscopy , 2012, IPCAI.

[86]  Adrien Bartoli,et al.  Shape-from-Polarization in laparoscopy , 2013, 2013 IEEE 10th International Symposium on Biomedical Imaging.

[87]  Lena Maier-Hein,et al.  Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery , 2013, Medical Image Anal..

[88]  N. Sugano Computer-Assisted Orthopaedic Surgery and Robotic Surgery in Total Hip Arthroplasty , 2013, Clinics in orthopedic surgery.