SenseCare: A Research Platform for Medical Image Informatics and Interactive 3D Visualization

Clinical research on smart healthcare has an increasing demand for intelligent and clinic-oriented medical image computing algorithms and platforms that support various applications. To this end, we have developed SenseCare research platform for smart healthcare, which is designed to boost translational research on intelligent diagnosis and treatment planning in various clinical scenarios. To facilitate clinical research with Artificial Intelligence (AI), SenseCare provides a range of AI toolkits for different tasks, including image segmentation, registration, lesion and landmark detection from various image modalities ranging from radiology to pathology. In addition, SenseCare is clinic-oriented and supports a wide range of clinical applications such as diagnosis and surgical planning for lung cancer, pelvic tumor, coronary artery disease, etc. SenseCare provides several appealing functions and features such as advanced 3D visualization, concurrent and efficient web-based access, fast data synchronization and high data security, multi-center deployment, support for collaborative research, etc. In this paper, we will present an overview of SenseCare as an efficient platform providing comprehensive toolkits and high extensibility for intelligent image analysis and clinical research in different application scenarios.

[1]  Ron Kikinis,et al.  3D Slicer , 2012, 2004 2nd IEEE International Symposium on Biomedical Imaging: Nano to Macro (IEEE Cat No. 04EX821).

[2]  Yike Guo,et al.  TensorLayer: A Versatile Library for Efficient Deep Learning Development , 2017, ACM Multimedia.

[3]  Jurgen Fripp,et al.  Robust inverse-consistent affine CT-MR registration in MRI-assisted and MRI-alone prostate radiation therapy , 2015, Medical Image Anal..

[4]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[5]  Dimitris N. Metaxas,et al.  Fully Automatic Segmentation Of Short-Axis Cardiac MRI Using Modified Deep Layer Aggregation , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[6]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[7]  Sébastien Ourselin,et al.  Fast free-form deformation using graphics processing units , 2010, Comput. Methods Programs Biomed..

[8]  Shengen Yan,et al.  Optimizing Network Performance for Distributed DNN Training on GPU Clusters: ImageNet/AlexNet Training in 1.5 Minutes , 2019, ArXiv.

[9]  Zhiqiang Hu,et al.  Accurate Nuclear Segmentation with Center Vector Encoding , 2019, IPMI.

[10]  Yuanyuan Chen,et al.  FocusNet: Imbalanced Large and Small Organ Segmentation with an End-to-End Deep Neural Network for Head and Neck CT Images , 2019, MICCAI.

[11]  Sébastien Ourselin,et al.  The NifTK software platform for image-guided interventions: platform overview and NiftyLink messaging , 2014, International Journal of Computer Assisted Radiology and Surgery.

[12]  Wei Liu,et al.  Towards Large-Scale Histopathological Image Analysis: Hashing-Based Image Retrieval , 2015, IEEE Transactions on Medical Imaging.

[13]  Jian Yang,et al.  Recurrent neural network for facial landmark detection , 2017, Neurocomputing.

[14]  Liang Zhao,et al.  Multi-resolution Path CNN with Deep Supervision for Intervertebral Disc Localization and Segmentation , 2019, MICCAI.

[15]  Shaoting Zhang,et al.  DeepIGeoS-V2: Deep Interactive Segmentation of Multiple Organs from Head and Neck Images with Lightweight CNNs , 2019, LABELS/HAL-MICCAI/CuRIOUS@MICCAI.

[16]  François Chollet,et al.  Keras: The Python Deep Learning library , 2018 .

[17]  Ben Glocker,et al.  DLTK: State of the Art Reference Implementations for Deep Learning on Medical Images , 2017, ArXiv.

[18]  Sebastien Ourselin,et al.  Automatic Segmentation of Vestibular Schwannoma from T2-Weighted MRI by Deep Spatial Attention with Hardness-Weighted Loss , 2019, MICCAI.

[19]  Henning Müller,et al.  Large‐scale retrieval for medical image analytics: A comprehensive review , 2018, Medical Image Anal..

[20]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Peng Liu,et al.  CFEA: Collaborative Feature Ensembling Adaptation for Domain Adaptation in Unsupervised Optic Disc and Cup Segmentation , 2019, MICCAI.

[22]  Max A. Viergever,et al.  elastix: A Toolbox for Intensity-Based Medical Image Registration , 2010, IEEE Transactions on Medical Imaging.

[23]  Hai Su,et al.  High-throughput histopathological image analysis via robust cell segmentation and hashing , 2015, Medical Image Anal..

[24]  Harri Valpola,et al.  Weight-averaged consistency targets improve semi-supervised deep learning results , 2017, ArXiv.

[25]  Max A. Viergever,et al.  A Recurrent CNN for Automatic Detection and Classification of Coronary Artery Plaque and Stenosis in Coronary CT Angiography , 2018, IEEE Transactions on Medical Imaging.

[26]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[27]  Jun Fu,et al.  Dual Attention Network for Scene Segmentation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[29]  Parashkev Nachev,et al.  Computer Methods and Programs in Biomedicine NiftyNet: a deep-learning platform for medical imaging , 2022 .

[30]  Christoph Meinel,et al.  Deep Learning for Medical Image Analysis , 2018, Journal of Pathology Informatics.

[31]  Jun Zhao,et al.  Vertebrae Identification and Localization Utilizing Fully Convolutional Networks and a Hidden Markov Model , 2020, IEEE Transactions on Medical Imaging.

[32]  Xinglong Liu,et al.  Pulmonary Vessel Segmentation Based on Orthogonal Fused U-Net++ of Chest CT Images , 2019, MICCAI.

[33]  Dimitris N. Metaxas,et al.  Improving Nuclei/Gland Instance Segmentation in Histopathology Images by Full Resolution Neural Network and Spatial Constrained Loss , 2019, MICCAI.

[34]  Alison Q. O'Neil,et al.  Attaining Human-Level Performance with Atlas Location Autocontext for Anatomical Landmark Detection in 3D CT Data , 2018, ECCV Workshops.

[35]  Klaus H. Maier-Hein,et al.  The Medical Imaging Interaction Toolkit: challenges and advances , 2013, International Journal of Computer Assisted Radiology and Surgery.

[36]  William J. Schroeder,et al.  The Visualization Toolkit , 2005, The Visualization Handbook.

[37]  Sébastien Ourselin,et al.  Automatic Brain Tumor Segmentation Based on Cascaded Convolutional Neural Networks With Uncertainty Estimation , 2019, Front. Comput. Neurosci..

[38]  Dimitris N. Metaxas,et al.  Collaborative Multi-agent Learning for MR Knee Articular Cartilage Segmentation , 2019, MICCAI.