Automated Anatomic Labeling Architecture for Content Discovery in Medical Imaging Repositories

The combination of textual data with visual features is known to enhance medical image search capabilities. However, the most advanced imaging archives today only index the studies’ available meta-data, often containing limited amounts of clinically useful information. This work proposes an anatomic labeling architecture, integrated with an open source archive software, for improved multimodal content discovery in real-world medical imaging repositories. The proposed solution includes a technical specification for classifiers in an extensible medical imaging archive, a classification database for querying over the extracted information, and a set of proof-of-concept convolutional neural network classifiers for identifying the presence of organs in computed tomography scans. The system automatically extracts the anatomic region features, which are saved in the proposed database for later consumption by multimodal querying mechanisms. The classifiers were evaluated with cross-validation, yielding a best F1-score of 96% and an average accuracy of 97%. We expect these capabilities to become common-place in production environments in the future, as automated detection solutions improve in terms of accuracy, computational performance, and interoperability.

[1]  Thomas Martin Deserno,et al.  A monohierarchical multiaxial classification code for medical images in content-based retrieval , 2002, Proceedings IEEE International Symposium on Biomedical Imaging.

[2]  José Luís Oliveira,et al.  Indexing and retrieving DICOM data in disperse and unstructured archives , 2008, International Journal of Computer Assisted Radiology and Surgery.

[3]  Tiago Marques Godinho,et al.  A Multimodal Search Engine for Medical Imaging Studies , 2017, Journal of Digital Imaging.

[4]  Michael Kohnen,et al.  Quality of DICOM header information for image categorization , 2002, SPIE Medical Imaging.

[5]  David Dagan Feng,et al.  Content-Based Medical Image Retrieval: A Survey of Applications to Multidimensional and Multimodality Data , 2013, Journal of Digital Imaging.

[6]  RieglerMichael,et al.  From Annotation to Computer-Aided Diagnosis , 2017 .

[7]  Ronald M. Summers,et al.  Anatomy-specific classification of medical images using deep convolutional nets , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).

[8]  José Luís Oliveira,et al.  A RESTful Image Gateway for Multiple Medical Image Repositories , 2012, IEEE Transactions on Information Technology in Biomedicine.

[9]  Hong Liu,et al.  A two-step convolutional neural network based computer-aided detection scheme for automatically segmenting adipose tissue volume depicting on CT images , 2017, Comput. Methods Programs Biomed..

[10]  H. K. Huang,et al.  Integration of computer-aided diagnosis/detection (CAD) results in a PACS environment using CAD–PACS toolkit and DICOM SR , 2009, International Journal of Computer Assisted Radiology and Surgery.

[11]  David Dagan Feng,et al.  An Ensemble of Fine-Tuned Convolutional Neural Networks for Medical Image Classification , 2017, IEEE Journal of Biomedical and Health Informatics.

[12]  Hayit Greenspan,et al.  Content-Based Image Retrieval in Radiology: Current Status and Future Directions , 2010, Journal of Digital Imaging.

[13]  Frederico Valente,et al.  Content Based Retrieval Systems in a Clinical Context , 2013 .

[14]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[15]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[16]  Kunio Doi,et al.  Computer-aided diagnosis in medical imaging: Historical review, current status and future potential , 2007, Comput. Medical Imaging Graph..

[17]  Tiago Marques Godinho,et al.  Anatomy of an Extensible Open Source PACS , 2016, Journal of Digital Imaging.

[18]  Heinz-Otto Peitgen,et al.  Rapid image recognition of body parts scanned in computed tomography datasets , 2009, International Journal of Computer Assisted Radiology and Surgery.

[19]  Tat-Seng Chua,et al.  Learning from Collective Intelligence , 2016, ACM Trans. Multim. Comput. Commun. Appl..

[20]  Michael Riegler,et al.  From Annotation to Computer-Aided Diagnosis , 2017, ACM Trans. Multim. Comput. Commun. Appl..

[21]  Anders M. Dale,et al.  An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest , 2006, NeuroImage.

[22]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[23]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[24]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[25]  Xiangrong Zhou,et al.  AUTOMATED RECOGNITION OF HUMAN STRUCURE FROM TORSO CT IMAGES , 2004 .

[26]  Michael Blum,et al.  High-Throughput Classification of Radiographs Using Deep Convolutional Neural Networks , 2016, Journal of Digital Imaging.

[27]  Malay Kumar Kundu,et al.  Interactive radiographic image retrieval system , 2017, Comput. Methods Programs Biomed..

[28]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[29]  Georg Langs,et al.  A Visual Information Retrieval System for Radiology Reports and the Medical Literature , 2014, MMM.

[30]  José Luís Oliveira,et al.  Semantic Search over DICOM Repositories , 2014, 2014 IEEE International Conference on Healthcare Informatics.

[31]  Daniel L. Rubin,et al.  On combining image-based and ontological semantic dissimilarities for medical image retrieval applications , 2014, Medical Image Anal..

[32]  Carlos Costa,et al.  Extensible Architecture for Multimodal Information Retrieval in Medical Imaging Archives , 2016, 2016 12th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS).

[33]  Daniel L Rubin,et al.  Content-based image retrieval in radiology: analysis of variability in human perception of similarity , 2015, Journal of medical imaging.

[34]  Frederico Valente,et al.  Dicoogle, a Pacs Featuring Profiled Content Based Image Retrieval , 2013, PloS one.