Machine Learning Applications of Surgical Imaging for the Diagnosis and Treatment of Spine Disorders: Current State of the Art

Recent developments in machine learning (ML) methods demonstrate unparalleled potential for application in the spine. The ability for ML to provide diagnostic faculty, produce novel insights from existing capabilities, and augment or accelerate elements of surgical planning and decision making at levels equivalent or superior to humans will tremendously benefit spine surgeons and patients alike. In this review, we aim to provide a clinically relevant outline of ML-based technology in the contexts of spinal deformity, degeneration, and trauma, as well as an overview of commercial-level and precommercial-level surgical assist systems and decisional support tools. Furthermore, we briefly discuss potential applications of generative networks before highlighting some of the limitations of ML applications. We conclude that ML in spine imaging represents a significant addition to the neurosurgeon's armamentarium—it has the capacity to directly address and manifest clinical needs and improve diagnostic and procedural quality and safety—but is yet subject to challenges that must be addressed before widespread implementation.

[1]  Kris B. Siemionow,et al.  Autonomous lumbar spine pedicle screw planning using machine learning: A validation study , 2021, Journal of craniovertebral junction & spine.

[2]  Jaemoon Yang,et al.  Deep Generative Adversarial Networks: Applications in Musculoskeletal Imaging. , 2021, Radiology. Artificial intelligence.

[3]  A. Wetter,et al.  Contrast agent dose reduction in computed tomography with deep learning using a conditional generative adversarial network , 2021, European Radiology.

[4]  A. Kyme,et al.  Efficient radiation dose reduction in whole-brain CT perfusion imaging using a 3D GAN: performance and clinical feasibility , 2021, Physics in medicine and biology.

[5]  Sharlene D. Newman,et al.  HydraNet: a multi-branch convolutional neural network architecture for MRI denoising , 2021, Medical Imaging.

[6]  Benoit M. Macq,et al.  Domain adversarial networks and intensity-based data augmentation for male pelvic organ segmentation in cone beam CT , 2021, Comput. Biol. Medicine.

[7]  W. Vandertop,et al.  Magnetic resonance imaging-based synthetic computed tomography of the lumbar spine for surgical planning: a clinical proof-of-concept. , 2021, Neurosurgical focus.

[8]  Matthew P. Lungren,et al.  Multimodal fusion with deep neural networks for leveraging CT imaging and electronic health record: a case-study in pulmonary embolism detection , 2020, Scientific Reports.

[9]  K. Lewandrowski,et al.  Artificial Intelligence Comparison of the Radiologist Report With Endoscopic Predictors of Successful Transforaminal Decompression for Painful Conditions of the Lumber Spine: Application of Deep Learning Algorithm Interpretation of Routine Lumbar Magnetic Resonance Imaging Scan , 2020, International Journal of Spine Surgery.

[10]  S. Inoue,et al.  Artificial intelligence for the detection of vertebral fractures on plain spinal radiography , 2020, Scientific Reports.

[11]  P. Newton,et al.  Machine Learning Predicts the 3D Outcomes of Adolescent Idiopathic Scoliosis Surgery Using Patient-Surgeon Specific Parameters. , 2020, Spine.

[12]  Tianfu Wang,et al.  GP-GAN: Brain tumor growth prediction using stacked 3D generative adversarial networks from longitudinal MR Images , 2020, Neural Networks.

[13]  Jong Chul Ye,et al.  Unpaired Deep Learning for Accelerated MRI Using Optimal Transport Driven CycleGAN , 2020, IEEE Transactions on Computational Imaging.

[14]  T. Solberg,et al.  DoseGAN: a generative adversarial network for synthetic dose prediction using attention-gated discrimination and generation , 2020, Scientific Reports.

[15]  Shirly Sundarsingh,et al.  Diagnosis of disc bulge and disc desiccation in lumbar MRI using concatenated shape and texture features with random forest classifier , 2020, Int. J. Imaging Syst. Technol..

[16]  K. Katchko,et al.  Augmented reality and artificial intelligence-assisted surgical navigation: Technique and cadaveric feasibility study , 2020, Journal of craniovertebral junction & spine.

[17]  T. Damron,et al.  Fracture risk assessment and clinical decision making for patients with metastatic bone disease , 2020, Journal of orthopaedic research : official publication of the Orthopaedic Research Society.

[18]  Yuhua Chen,et al.  MRI Super-Resolution with GAN and 3D Multi-Level DenseNet: Smaller, Faster, and Better , 2020, ArXiv.

[19]  Timothy D. Solberg,et al.  Attention-Aware Discrimination for MR-to-CT Image Translation Using Cycle-Consistent Generative Adversarial Networks. , 2020, Radiology. Artificial intelligence.

[20]  Cheng-Bin Jin,et al.  Spine Computed Tomography to Magnetic Resonance Image Synthesis Using Generative Adversarial Networks : A Preliminary Study , 2020, Journal of Korean Neurosurgical Society.

[21]  M. Matsumoto,et al.  An Application of Artificial Intelligence to Diagnostic Imaging of Spine Disease: Estimating Spinal Alignment From Moiré Images , 2019, Neurospine.

[22]  Victor E Staartjes,et al.  Data Mining in Spine Surgery: Leveraging Electronic Health Records for Machine Learning and Clinical Research , 2019, Neurospine.

[23]  Eun Kyung Choe,et al.  The exploration of feature extraction and machine learning for predicting bone density from simple spine X-ray images in a Korean population , 2019, Skeletal Radiology.

[24]  W. Tan,et al.  Development and validation of deep learning algorithms for scoliosis screening using back images , 2019, Communications Biology.

[25]  Fucang Jia,et al.  UC-GAN for MR to CT Image Synthesis , 2019, AIRT@MICCAI.

[26]  Jan Mikulka,et al.  Pediatric Spine Segmentation and Modeling Using Machine Learning , 2019, 2019 11th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT).

[27]  Hongming Shan,et al.  Super-resolution MRI and CT through GAN-CIRCLE , 2019, Developments in X-Ray Tomography XII.

[28]  María Teresa Martín-Valdivia,et al.  Strengths, Weaknesses, Opportunities, and Threats Analysis of Artificial Intelligence and Machine Learning Applications in Radiology. , 2019, Journal of the American College of Radiology : JACR.

[29]  Safdar N. Khan,et al.  Predictive modeling in spine surgery. , 2019, Annals of translational medicine.

[30]  Yaling Pan,et al.  Evaluation of a computer-aided method for measuring the Cobb angle on chest X-rays , 2019, European Spine Journal.

[31]  Quinlan D. Buchlak,et al.  Machine learning applications to clinical decision support in neurosurgery: an artificial intelligence augmented systematic review , 2019, Neurosurgical Review.

[32]  Aly A. Valliani,et al.  Automated Measurement of Lumbar Lordosis on Radiographs Using Machine Learning and Computer Vision , 2019, Global spine journal.

[33]  Afshin Gangi,et al.  Augmented reality and artificial intelligence-based navigation during percutaneous vertebroplasty: a pilot randomised clinical trial , 2019, European Spine Journal.

[34]  Benjamin S. Hopkins,et al.  Machine Learning for the Prediction of Cervical Spondylotic Myelopathy: A Post Hoc Pilot Study of 28 Participants. , 2019, World neurosurgery.

[35]  Irfan Essa,et al.  Artificial Intelligence for the Treatment of Lumbar Spondylolisthesis. , 2019, Neurosurgery clinics of North America.

[36]  Kit-Hang Lee,et al.  Respiratory Motion Correction in Abdominal MRI using a Densely Connected U-Net with GAN-guided Training , 2019, ArXiv.

[37]  Ran Harel,et al.  Augmented reality-assisted pedicle screw insertion: a cadaveric proof-of-concept study. , 2019, Journal of neurosurgery. Spine.

[38]  Cristian Lorenz,et al.  Machine learning for automated 3-dimensional segmentation of the spine and suggested placement of pedicle screws based on intraoperative cone-beam computer tomography. , 2019, Journal of neurosurgery. Spine.

[39]  H. Wilke,et al.  Fully automated radiological analysis of spinal disorders and deformities: a deep learning approach , 2019, European Spine Journal.

[40]  L. Floridi,et al.  Clinical applications of machine learning algorithms: beyond the black box , 2019, BMJ.

[41]  E. Salehi,et al.  Lumbar Spine Vertebral Compression Fracture Case Diagnosis Using Machine Learning Methods on CT images , 2019, 2019 4th International Conference on Pattern Recognition and Image Analysis (IPRIA).

[42]  Chii-Jeng Lin,et al.  Cobb Angle Measurement of Spine from X-Ray Images Using Convolutional Neural Network , 2019, Comput. Math. Methods Medicine.

[43]  A. Becker,et al.  Vertebral body insufficiency fractures: detection of vertebrae at risk on standard CT images using texture analysis and machine learning , 2018, European Radiology.

[44]  Shuo Li,et al.  Spine‐GAN: Semantic segmentation of multiple spinal structures , 2018, Medical Image Anal..

[45]  Jing Wang,et al.  Generating synthesized computed tomography (CT) from cone-beam computed tomography (CBCT) using CycleGAN for adaptive radiation therapy , 2018, Physics in medicine and biology.

[46]  Talha Iqbal,et al.  Generative Adversarial Network for Medical Images (MI-GAN) , 2018, Journal of Medical Systems.

[47]  Gang Yu,et al.  3D Path Planning for Anterior Spinal Surgery Based on CT images and Reinforcement Learning* , 2018, 2018 IEEE International Conference on Cyborg and Bionic Systems (CBS).

[48]  Paul Babyn,et al.  Generative Adversarial Network in Medical Imaging: A Review , 2018, Medical Image Anal..

[49]  Michael Söderman,et al.  Pedicle Screw Placement Using Augmented Reality Surgical Navigation With Intraoperative 3D Imaging , 2018, Spine.

[50]  Guang Li,et al.  CT Super-Resolution GAN Constrained by the Identical, Residual, and Cycle Learning Ensemble (GAN-CIRCLE) , 2018, IEEE Transactions on Medical Imaging.

[51]  Shuo Li,et al.  Automated comprehensive Adolescent Idiopathic Scoliosis assessment using MVC‐Net , 2018, Medical Image Anal..

[52]  Stefano Pedemonte,et al.  DeepSPINE: Automated Lumbar Vertebral Segmentation, Disc-level Designation, and Spinal Stenosis Grading Using Deep Learning , 2018, ArXiv.

[53]  Timothy C. Y. Chan,et al.  Automated Treatment Planning in Radiation Therapy using Generative Adversarial Networks , 2018, MLHC.

[54]  Michael Söderman,et al.  Feasibility and Accuracy of Thoracolumbar Minimally Invasive Pedicle Screw Placement With Augmented Reality Navigation Technology , 2018, Spine.

[55]  Marcus Hacker,et al.  Personalizing Medicine Through Hybrid Imaging and Medical Big Data Analysis , 2018, Front. Phys..

[56]  Ahmed Hosny,et al.  Artificial intelligence in radiology , 2018, Nature Reviews Cancer.

[57]  M. Shrime,et al.  Degenerative Lumbar Spine Disease: Estimating Global Incidence and Worldwide Volume , 2018, Global spine journal.

[58]  B. Cunningham,et al.  Middle-Column Gap Balancing and Middle-Column Mismatch in Spinal Reconstructive Surgery , 2018, International Journal of Spine Surgery.

[59]  Yufeng Zhang,et al.  Computer-Aided Cobb Measurement Based on Automatic Detection of Vertebral Slopes Using Deep Neural Network , 2017, Int. J. Biomed. Imaging.

[60]  Shuo Li,et al.  Automatic Landmark Estimation for Adolescent Idiopathic Scoliosis Assessment Using BoostNet , 2017, MICCAI.

[61]  Won-Ki Jeong,et al.  Compressed Sensing MRI Reconstruction Using a Generative Adversarial Network With a Cyclic Loss , 2017, IEEE Transactions on Medical Imaging.

[62]  Xuanqin Mou,et al.  Low-Dose CT Image Denoising Using a Generative Adversarial Network With Wasserstein Distance and Perceptual Loss , 2017, IEEE Transactions on Medical Imaging.

[63]  Thomas Neff,et al.  Generative Adversarial Network based Synthesis for Supervised Medical Image Segmentation , 2017 .

[64]  Ronald M. Summers,et al.  Vertebral Body Compression Fractures and Bone Density: Automated Detection and Classification on CT Images. , 2017, Radiology.

[65]  Ganapathy Krishnamurthi,et al.  Generative adversarial networks for brain lesion detection , 2017, Medical Imaging.

[66]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[67]  Alejandro F. Frangi,et al.  Intervertebral disc classification by its degree of degeneration from T2-weighted magnetic resonance images , 2016, European Spine Journal.

[68]  Ronald M. Summers,et al.  Deep convolutional networks for automated detection of posterior-element fractures on spine CT , 2016, SPIE Medical Imaging.

[69]  Estanislao Arana,et al.  Semiautomatic computer-aided classification of degenerative lumbar spine disease in magnetic resonance imaging , 2015, Comput. Biol. Medicine.

[70]  Samer Adeeb,et al.  Monitoring for idiopathic scoliosis curve progression using surface topography asymmetry analysis of the torso in adolescents. , 2015, The spine journal : official journal of the North American Spine Society.

[71]  Ayse Betül Oktay,et al.  Computer aided diagnosis of degenerative intervertebral disc diseases from lumbar MR images , 2014, Comput. Medical Imaging Graph..

[72]  Christopher Nimsky,et al.  Robust Detection and Segmentation for Diagnosis of Vertebral Diseases Using Routine MR Images , 2014, Comput. Graph. Forum.

[73]  Gabor Fichtinger,et al.  MR-Guided Vertebroplasty With Augmented Reality Image Overlay Navigation , 2014, CardioVascular and Interventional Radiology.

[74]  Manabu Ito,et al.  A novel 3D guidance system using augmented reality for percutaneous vertebroplasty: technical note. , 2013, Journal of neurosurgery. Spine.

[75]  Vipin Chaudhary,et al.  Compression fracture diagnosis in lumbar: a clinical CAD system , 2013, International Journal of Computer Assisted Radiology and Surgery.

[76]  P. Eysel,et al.  Measuring procedures to determine the Cobb angle in idiopathic scoliosis: a systematic review , 2013, European Spine Journal.

[77]  Shreyasee Amin,et al.  Assessing fracture risk using gradient boosting machine (GBM) models , 2012, Journal of bone and mineral research : the official journal of the American Society for Bone and Mineral Research.

[78]  Vipin Chaudhary,et al.  Composite features for automatic diagnosis of intervertebral disc herniation from lumbar MRI , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[79]  Vipin Chaudhary,et al.  Computer-aided diagnosis for lumbar mri using heterogeneous classifiers , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[80]  Merisandra Côrtes de Mattos,et al.  Análise computacional para auxílio ao diagnóstico de osteoartrite de coluna lombar baseado em redes neurais artificiais , 2011 .

[81]  Yuanyuan Wang,et al.  Automatic Cobb Measurement of Scoliosis Based on Fuzzy Hough Transform with Vertebral Shape Prior , 2009, Journal of Digital Imaging.

[82]  Lena Costaridou,et al.  Atlas-Based Segmentation of Degenerated Lumbar Intervertebral Discs From MR Images of the Spine , 2009, IEEE Transactions on Biomedical Engineering.

[83]  C. Bach,et al.  Inter- and intraobserver reliability assessment of the Cobb angle: manual versus digital measurement tools , 2007, European Spine Journal.

[84]  Farida Cheriet,et al.  Three-Dimensional Classification of Spinal Deformities Using Fuzzy Clustering , 2006, Spine.

[85]  Farida Cheriet,et al.  Prediction of anterior scoliotic spinal curve from trunk surface using support vector regression , 2005, Eng. Appl. Artif. Intell..

[86]  F. Schwab,et al.  Adult Scoliosis: Prevalence, SF-36, and Nutritional Parameters in an Elderly Volunteer Population , 2005, Spine.

[87]  H Labelle,et al.  Estimation of Spinal Deformity in Scoliosis From Torso Surface Cross Sections , 2001, Spine.

[88]  Joseph Y. Cheng,et al.  Deep Generative Adversarial Networks for Compressed Sensing ( GANCS ) Automates MRI , 2017 .

[89]  Hayriye Korkmaz,et al.  Rule-based fuzzy classifier for spinal deformities. , 2014, Bio-medical materials and engineering.

[90]  D. Alexandru,et al.  Evaluation and management of vertebral compression fractures. , 2012, The Permanente journal.

[91]  Nelson G. Durdle,et al.  A support vector machines classifier to assess the severity of idiopathic scoliosis from surface topography , 2006, IEEE Transactions on Information Technology in Biomedicine.