Machine learning applications in imaging analysis for patients with pituitary tumors: a review of the current literature and future directions

Purpose To provide an overview of fundamental concepts in machine learning (ML), review the literature on ML applications in imaging analysis of pituitary tumors for the last 10 years, and highlight the future directions on potential applications of ML for pituitary tumor patients. Method We presented an overview of the fundamental concepts in ML, its various stages used in healthcare, and highlighted the key components typically present in an imaging-based tumor analysis pipeline. A search was conducted across four databases (PubMed, Ovid, Embase, and Google Scholar) to gather research articles from the past 10 years (2009-2019) involving imaging related to pituitary tumor and ML. We grouped the studies by imaging modalities and analyzed the ML tasks in terms of the data inputs, reference standards, methodologies, and limitations. Results Of the 16 studies included in our analysis, 10 appeared in 2018–2019. Most of the studies utilized retrospective data and followed a semi-automatic ML pipeline. The studies included use of magnetic resonance imaging (MRI), facial photographs, surgical microscopic video, spectrometry, and spectroscopy imaging. The objectives of the studies covered 14 distinct applications and majority of the studies addressed a binary classification problem. Only five of the 11 MRI-based studies had an external validation or a holdout set to test the performance of a final trained model. Conclusion Through our concise evaluation and comparison of the studies using the concepts presented, we highlight future directions so that potential ML applications using different imaging modalities can be developed to benefit the clinical care of pituitary tumor patients.

[1]  S. Melmed Pathogenesis of pituitary tumors , 2011, Nature Reviews Endocrinology.

[2]  N. Kocer,et al.  Preoperative evaluation of tumour consistency in pituitary macroadenomas: a machine learning-based histogram analysis on conventional T2-weighted MRI , 2019, Neuroradiology.

[3]  Andre Esteva,et al.  A guide to deep learning in healthcare , 2019, Nature Medicine.

[4]  I. Norton,et al.  MALDI mass spectrometry imaging analysis of pituitary adenomas for near-real-time tumor delineation , 2015, Proceedings of the National Academy of Sciences.

[5]  George Lee,et al.  Image analysis and machine learning in digital pathology: Challenges and opportunities , 2016, Medical Image Anal..

[6]  Xiaozhi Yu,et al.  Effects of MRI scanner parameters on breast cancer radiomics , 2017, Expert Syst. Appl..

[7]  M. Mazurowski,et al.  Management of Thyroid Nodules Seen on US Images: Deep Learning May Match Performance of Radiologists. , 2019, Radiology.

[8]  K. Kovacs,et al.  Recovery Room Cortisol Predicts Long-Term Glucocorticoid Need After Transsphenoidal Surgery for Pituitary Tumors , 2018, Neurosurgery.

[9]  Dmitrii Bychkov,et al.  Deep learning based tissue analysis predicts outcome in colorectal cancer , 2018, Scientific Reports.

[10]  Christopher Nimsky,et al.  Template-Cut: A Pattern-Based Segmentation Paradigm , 2012, Scientific Reports.

[11]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[12]  E. Krenning,et al.  Neuroendocrine tumours: the role of imaging for diagnosis and therapy , 2014, Nature Reviews Endocrinology.

[13]  E. Knosp,et al.  Invasion of the cavernous sinus space in pituitary adenomas: endoscopic verification and its correlation with an MRI-based classification. , 2015, Journal of neurosurgery.

[14]  Thomas Frauenfelder,et al.  Influence of inter-observer delineation variability on radiomics stability in different tumor sites , 2018, Acta oncologica.

[15]  Y. Lui,et al.  State of the Art: Machine Learning Applications in Glioma Imaging. , 2019, AJR. American journal of roentgenology.

[16]  Erhan Guven,et al.  A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection , 2016, IEEE Communications Surveys & Tutorials.

[17]  V. Staartjes,et al.  Neural network-based identification of patients at high risk for intraoperative cerebrospinal fluid leaks in endoscopic pituitary surgery. , 2020, Journal of neurosurgery.

[18]  P. M. Ameer,et al.  Brain tumor classification using deep CNN features via transfer learning , 2019, Comput. Biol. Medicine.

[19]  Antonio Di Ieva,et al.  Aggressive pituitary adenomas—diagnosis and emerging treatments , 2014, Nature Reviews Endocrinology.

[20]  V. Staartjes,et al.  Utility of deep neural networks in predicting gross-total resection after transsphenoidal surgery for pituitary adenoma: a pilot study. , 2018, Neurosurgical focus.

[21]  Gianpaolo Francesco Trotta,et al.  Computer vision and deep learning techniques for pedestrian detection and tracking: A survey , 2018, Neurocomputing.

[22]  F. Sardanelli,et al.  Artificial intelligence in medical imaging: threat or opportunity? Radiologists again at the forefront of innovation in medicine , 2018, European Radiology Experimental.

[23]  Nidan Qiao A systematic review on machine learning in sellar region diseases: quality and reporting items , 2019, Endocrine connections.

[24]  X. Bao,et al.  Preoperative Noninvasive Radiomics Approach Predicts Tumor Consistency in Patients With Acromegaly: Development and Multicenter Prospective Validation , 2019, Front. Endocrinol..

[25]  Mateusz Buda,et al.  Deep learning in radiology: An overview of the concepts and a survey of the state of the art with focus on MRI , 2018, Journal of magnetic resonance imaging : JMRI.

[26]  B. Erickson,et al.  Machine Learning for Medical Imaging. , 2017, Radiographics : a review publication of the Radiological Society of North America, Inc.

[27]  Julian Togelius,et al.  Deep Learning for Video Game Playing , 2017, IEEE Transactions on Games.

[28]  M. Buchfelder,et al.  Intraoperative Magnetic Resonance Imaging for Pituitary Adenomas. , 2016, Frontiers of hormone research.

[29]  John A. Evans,et al.  Combined high-field intraoperative magnetic resonance imaging and endoscopy increase extent of resection and progression-free survival for pituitary adenomas , 2015, Pituitary.

[30]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[31]  L. Cavallo,et al.  Prediction of high proliferative index in pituitary macroadenomas using MRI-based radiomics and machine learning , 2019, Neuroradiology.

[32]  D. Dong,et al.  Non-invasive radiomics approach potentially predicts non-functioning pituitary adenomas subtypes before surgery , 2018, European Radiology.

[33]  Qinghua Zhao,et al.  Brain tumor classification for MR images using transfer learning and fine-tuning , 2019, Comput. Medical Imaging Graph..

[34]  G. Starck,et al.  Visual pathway impairment by pituitary adenomas: quantitative diagnostics by diffusion tensor imaging. , 2017, Journal of neurosurgery.

[35]  Maciej A Mazurowski,et al.  Breast cancer MRI radiomics: An overview of algorithmic features and impact of inter‐reader variability in annotating tumors , 2018, Medical physics.

[36]  Qianjin Feng,et al.  Enhanced Performance of Brain Tumor Classification via Tumor Region Augmentation and Partition , 2015, PloS one.

[37]  T. V. Vliet Vlieland,et al.  Work disability and its determinants in patients with pituitary tumor-related disease , 2018, Pituitary.

[38]  Brian B. Avants,et al.  The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) , 2015, IEEE Transactions on Medical Imaging.

[39]  Parisa Rashidi,et al.  Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis , 2017, IEEE Journal of Biomedical and Health Informatics.

[40]  N. Kocer,et al.  Predicting response to somatostatin analogues in acromegaly: machine learning-based high-dimensional quantitative texture analysis on T2-weighted MRI , 2018, European Radiology.

[41]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[42]  C. Soh,et al.  Assessing size of pituitary adenomas: a comparison of qualitative and quantitative methods on MR , 2016, Acta neurochirurgica.

[43]  Chan Hee Moon,et al.  Visual prognostic value of optical coherence tomography and photopic negative response in chiasmal compression. , 2011, Investigative ophthalmology & visual science.

[44]  Pierre Jannin,et al.  Automatic Phases Recognition in Pituitary Surgeries by Microscope Images Classification , 2010, IPCAI.

[45]  Enrico Pellegrini,et al.  Machine learning of neuroimaging for assisted diagnosis of cognitive impairment and dementia: A systematic review , 2018, Alzheimer's & dementia.

[46]  Shereen Ezzat,et al.  The prevalence of pituitary adenomas , 2004, Cancer.

[47]  Andrew Y. Ng,et al.  CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning , 2017, ArXiv.

[48]  Shai Ben-David,et al.  Understanding Machine Learning: From Theory to Algorithms , 2014 .

[49]  Min Chen,et al.  Disease Prediction by Machine Learning Over Big Data From Healthcare Communities , 2017, IEEE Access.

[50]  Edmund Koch,et al.  Label-free differentiation of human pituitary adenomas by FT-IR spectroscopic imaging , 2012, Analytical and Bioanalytical Chemistry.

[51]  Sung-Hoon Ahn,et al.  Smart Machining Process Using Machine Learning: A Review and Perspective on Machining Industry , 2018, International Journal of Precision Engineering and Manufacturing-Green Technology.

[52]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[53]  Ali Kashif Bashir,et al.  Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) , 2013, ICIRA 2013.

[54]  Joey Nicholson,et al.  Automated Segmentation of Tissues Using CT and MRI: A Systematic Review. , 2019, Academic radiology.

[55]  Maite Frutos-Pascual,et al.  Review of the Use of AI Techniques in Serious Games: Decision Making and Machine Learning , 2017, IEEE Transactions on Computational Intelligence and AI in Games.

[56]  M. Cusimano,et al.  Endoscopic versus microscopic approach for surgical treatment of acromegaly , 2015, Neurosurgical Review.

[57]  P. Lambin,et al.  Defining the biological basis of radiomic phenotypes in lung cancer , 2017, eLife.

[58]  C. Jack,et al.  Alzheimer's Disease Neuroimaging Initiative (ADNI) , 2010, Neurology.

[59]  A. Ribeiro-Oliveira,et al.  Non-Functioning Pituitary Adenomas , 2018 .

[60]  Nikolaos Doulamis,et al.  Deep Learning for Computer Vision: A Brief Review , 2018, Comput. Intell. Neurosci..

[61]  Andriy Fedorov,et al.  Computational Radiomics System to Decode the Radiographic Phenotype. , 2017, Cancer research.

[62]  L. Deangelis,et al.  Glioblastoma and Other Malignant Gliomas , 2013 .

[63]  José García Rodríguez,et al.  A survey on deep learning techniques for image and video semantic segmentation , 2018, Appl. Soft Comput..

[64]  Laurence T. Yang,et al.  A survey on deep learning for big data , 2018, Inf. Fusion.

[65]  M. Cusimano,et al.  Extent of Surgery in Endoscopic Transsphenoidal Skull Base Approaches and the Effects on Sinonasal Morbidity , 2018, American journal of rhinology & allergy.

[66]  Xiangyi Kong,et al.  Automatic Detection of Acromegaly From Facial Photographs Using Machine Learning Methods , 2017, EBioMedicine.

[67]  C. Soh,et al.  Correction to: Assessing size of pituitary adenomas: a comparison of qualitative and quantitative methods on MR , 2016, Acta Neurochirurgica.

[68]  Maciej A Mazurowski,et al.  A machine learning approach to radiogenomics of breast cancer: a study of 922 subjects and 529 DCE-MRI features , 2018, British Journal of Cancer.

[69]  O. Ijare,et al.  Ex Vivo1H NMR study of pituitary adenomas to differentiate various immunohistochemical subtypes , 2019, Scientific Reports.

[70]  Rolf P. Würtz,et al.  A novel approach to the detection of acromegaly: accuracy of diagnosis by automatic face classification. , 2011, The Journal of clinical endocrinology and metabolism.

[71]  Suhuai Luo,et al.  Automatic Alzheimer’s Disease Recognition from MRI Data Using Deep Learning Method , 2017 .

[72]  B. Xing,et al.  Prediction of Recurrence after Transsphenoidal Surgery for Cushing’s Disease: The Use of Machine Learning Algorithms , 2019, Neuroendocrinology.

[73]  Jinzhong Yang,et al.  Measuring Computed Tomography Scanner Variability of Radiomics Features , 2015, Investigative radiology.

[74]  Daniel Fabbri,et al.  Deep learning for brain tumor classification , 2017, Medical Imaging.

[75]  Baris Turkbey,et al.  Radiomics and radiogenomics of prostate cancer , 2018, Abdominal Radiology.

[76]  Gary S. Collins,et al.  Reporting of artificial intelligence prediction models , 2019, The Lancet.

[77]  Bradley A. Otto,et al.  Application of image guidance in pituitary surgery , 2012, Surgical neurology international.

[78]  Ladislav Hluchý,et al.  Machine Learning and Deep Learning frameworks and libraries for large-scale data mining: a survey , 2019, Artificial Intelligence Review.

[79]  Paul Kinahan,et al.  Radiomics: Images Are More than Pictures, They Are Data , 2015, Radiology.

[80]  Erik Cambria,et al.  Recent Trends in Deep Learning Based Natural Language Processing , 2017, IEEE Comput. Intell. Mag..

[81]  A. Vighetto,et al.  Predicting visual outcome after treatment of pituitary adenomas with optical coherence tomography. , 2009, American journal of ophthalmology.

[82]  Marcy G Lake,et al.  Pituitary adenomas: an overview. , 2013, American family physician.

[83]  P. Cao,et al.  Endoscopic Versus Microscopic Transsphenoidal Surgery in the Treatment of Pituitary Adenoma: A Systematic Review and Meta-Analysis. , 2017, World neurosurgery.

[84]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[85]  N. Oyesiku,et al.  Congress of Neurological Surgeons Systematic Review and Evidence-Based Guideline on Preoperative Imaging Assessment of Patients With Suspected Nonfunctioning Pituitary Adenomas. , 2016, Neurosurgery.

[86]  Jie Tian,et al.  Preoperative prediction of cavernous sinus invasion by pituitary adenomas using a radiomics method based on magnetic resonance images , 2018, European Radiology.

[87]  L. Deangelis,et al.  Glioblastoma and other malignant gliomas: a clinical review. , 2013, JAMA.

[88]  M. Buchfelder,et al.  Diagnosis and Management of Pituitary Carcinomas , 2005 .