Bladder segmentation based on deep learning approaches: current limitations and lessons

Precise determination and assessment of bladder cancer (BC) extent of muscle invasion involvement guides proper risk stratification and personalized therapy selection. In this context, segmentation of both bladder walls and cancer are of pivotal importance, as it provides invaluable information to stage the primary tumor. Hence, multiregion segmentation on patients presenting with symptoms of bladder tumors using deep learning heralds a new level of staging accuracy and prediction of the biologic behavior of the tumor. Nevertheless, despite the success of these models in other medical problems, progress in multiregion bladder segmentation is still at a nascent stage, with just a handful of works tackling a multiregion scenario. Furthermore, most existing approaches systematically follow prior literature in other clinical problems, without casting a doubt on the validity of these methods on bladder segmentation, which may present different challenges. Inspired by this, we provide an in-depth look at bladder cancer segmentation using deep learning models. The critical determinants for accurate differentiation of muscle invasive disease, current status of deep learning based bladder segmentation, lessons and limitations of prior work are highlighted.

[1]  Jose Dolz,et al.  Deep Interpretable Classification and Weakly-Supervised Segmentation of Histology Images via Max-Min Uncertainty , 2020, IEEE Transactions on Medical Imaging.

[2]  Jose Dolz,et al.  Multi-Scale Self-Guided Attention for Medical Image Segmentation , 2021, IEEE Journal of Biomedical and Health Informatics.

[3]  V. Panebianco,et al.  Staging of bladder cancer with multiparametric MRI. , 2020, The British journal of radiology.

[4]  Ismail Ben Ayed,et al.  Source-Relaxed Domain Adaptation for Image Segmentation , 2020, MICCAI.

[5]  C. Catalano,et al.  Overview of VI-RADS in Bladder Cancer. , 2020, AJR. American journal of roentgenology.

[6]  Jose Dolz,et al.  Manifold-driven Attention Maps for Weakly Supervised Segmentation , 2020, ArXiv.

[7]  K. Hammouda,et al.  A 3D CNN with a Learnable Adaptive Shape Prior for Accurate Segmentation of Bladder Wall Using MR Images , 2020, 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI).

[8]  E. Sala,et al.  MRI of Bladder Cancer: Local and Nodal Staging. , 2020, Journal of magnetic resonance imaging : JMRI.

[9]  Jose Dolz,et al.  Bounding boxes for weakly supervised segmentation: Global constraints get close to full supervision , 2020, MIDL.

[10]  Pheng Ann Heng,et al.  Unpaired Multi-Modal Segmentation via Knowledge Distillation , 2020, IEEE Transactions on Medical Imaging.

[11]  Jizong Peng,et al.  Discretely-constrained deep network for weakly supervised segmentation , 2019, Neural Networks.

[12]  Septimiu E. Salcudean,et al.  Reducing the Hausdorff Distance in Medical Image Segmentation With Convolutional Neural Networks , 2019, IEEE Transactions on Medical Imaging.

[13]  Pablo Piantanida,et al.  On Direct Distribution Matching for Adapting Segmentation Networks , 2019, MIDL.

[14]  F. Khalifa,et al.  A Deep Learning-Based Approach for Accurate Segmentation of Bladder Wall using MR Images , 2019, 2019 IEEE International Conference on Imaging Systems and Techniques (IST).

[15]  C. Catalano,et al.  Prospective Assessment of Vesical Imaging Reporting and Data System (VI-RADS) and Its Clinical Impact on the Management of High-risk Non-muscle-invasive Bladder Cancer Patients Candidate for Repeated Transurethral Resection. , 2019, European urology.

[16]  Adel Said Elmaghraby,et al.  A CNN-Based Framework for Bladder Wall Segmentation Using MRI , 2019, 2019 Fifth International Conference on Advances in Biomedical Engineering (ICABME).

[17]  Jose Dolz,et al.  Constrained domain adaptation for segmentation , 2019, MICCAI.

[18]  Cheng Chen,et al.  PnP-AdaNet: Plug-and-Play Adversarial Domain Adaptation Network at Unpaired Cross-Modality Cardiac Segmentation , 2019, IEEE Access.

[19]  K. Harada,et al.  Diagnostic Accuracy and Interobserver Agreement for the Vesical Imaging-Reporting and Data System for Muscle-invasive Bladder Cancer: A Multireader Validation Study. , 2019, European urology.

[20]  Eric Granger,et al.  Curriculum semi-supervised segmentation , 2019, MICCAI.

[21]  Guoping Qiu,et al.  Bladder Cancer Multi-Class Segmentation in MRI With Pyramid-In-Pyramid Network , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[22]  Lubomir M. Hadjiiski,et al.  U‐Net based deep learning bladder segmentation in CT urography , 2019, Medical physics.

[23]  C. Catalano,et al.  Multiparametric MRI of the bladder: inter-observer agreement and accuracy with the Vesical Imaging-Reporting and Data System (VI-RADS) at a single reference center , 2019, European Radiology.

[24]  Lubomir M. Hadjiiski,et al.  2D and 3D bladder segmentation using U-Net-based deep-learning , 2019, Medical Imaging.

[25]  Christophe De Vleeschouwer,et al.  Using planning CTs to enhance CNN-based bladder segmentation on cone beam CT , 2019, Medical Imaging.

[26]  Gang Li,et al.  Benchmark on Automatic Six-Month-Old Infant Brain Segmentation Algorithms: The iSeg-2017 Challenge , 2019, IEEE Transactions on Medical Imaging.

[27]  Lubomir M. Hadjiiski,et al.  Deep‐learning convolutional neural network: Inner and outer bladder wall segmentation in CT urography , 2019, Medical physics.

[28]  Jose Dolz,et al.  Boundary loss for highly unbalanced segmentation , 2018, MIDL.

[29]  Eric Granger,et al.  Constrained‐CNN losses for weakly supervised segmentation☆ , 2018, Medical Image Anal..

[30]  Jing Yuan,et al.  HyperDense-Net: A Hyper-Densely Connected CNN for Multi-Modal Image Segmentation , 2018, IEEE Transactions on Medical Imaging.

[31]  Antonio Pepe,et al.  PET-Train: Automatic Ground Truth Generation from PET Acquisitions for Urinary Bladder Segmentation in CT Images using Deep Learning , 2018, 2018 11th Biomedical Engineering International Conference (BMEiCON).

[32]  Christophe De Vleeschouwer,et al.  Contour Propagation in CT Scans with Convolutional Neural Networks , 2018, ACIVS.

[33]  Jose Dolz,et al.  IVD-Net: Intervertebral disc localization and segmentation in MRI with a multi-modal UNet , 2018, CSI@MICCAI.

[34]  R. Huddart,et al.  Multiparametric Magnetic Resonance Imaging for Bladder Cancer: Development of VI-RADS (Vesical Imaging-Reporting And Data System). , 2018, European urology.

[35]  Tanveer F. Syeda-Mahmood,et al.  3D Segmentation with Exponential Logarithmic Loss for Highly Unbalanced Object Sizes , 2018, MICCAI.

[36]  P. Bhosale,et al.  Immunotherapy and the role of imaging , 2018, Cancer.

[37]  Jing Yuan,et al.  Multiregion segmentation of bladder cancer structures in MRI with progressive dilated convolutional networks , 2018, Medical physics.

[38]  Xin Yang,et al.  Deep Learning Techniques for Automatic MRI Cardiac Multi-Structures Segmentation and Diagnosis: Is the Problem Solved? , 2018, IEEE Transactions on Medical Imaging.

[39]  Ismail Ben Ayed,et al.  On Regularized Losses for Weakly-supervised CNN Segmentation , 2018, ECCV.

[40]  Fugen Zhou,et al.  Automatic bladder segmentation from CT images using deep CNN and 3D fully connected CRF-RNN , 2018, International Journal of Computer Assisted Radiology and Surgery.

[41]  Jin Wang,et al.  The Diagnostic Value of MR Imaging in Differentiating T Staging of Bladder Cancer: A Meta-Analysis. , 2017, Radiology.

[42]  Bo Du,et al.  Shape prior constrained PSO model for bladder wall MRI segmentation , 2017, Neurocomputing.

[43]  K. Nikolaou,et al.  Imaging response assessment of immunotherapy in patients with renal cell and urothelial carcinoma , 2017, Current opinion in urology.

[44]  G. Guazzoni,et al.  Role of Restaging Transurethral Resection for T1 Non-muscle invasive Bladder Cancer: A Systematic Review and Meta-analysis. , 2017, European urology focus.

[45]  Jose Dolz,et al.  3D fully convolutional networks for subcortical segmentation in MRI: A large-scale study , 2016, NeuroImage.

[46]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Dimos Baltas,et al.  Esophagus segmentation in CT via 3D fully convolutional neural network and random walk , 2017, Medical physics.

[48]  J. Cho,et al.  Diagnostic performance of MRI for prediction of muscle-invasiveness of bladder cancer: A systematic review and meta-analysis. , 2017, European journal of radiology.

[49]  Jing Yuan,et al.  Simultaneous Segmentation of Multiple Regions in 3D Bladder MRI by Efficient Convex Optimization of Coupled Surfaces , 2017, ICIG.

[50]  J. Tavares,et al.  A versatile method for bladder segmentation in computed tomography two-dimensional images under adverse conditions , 2017, Proceedings of the Institution of Mechanical Engineers. Part H, Journal of engineering in medicine.

[51]  Sébastien Ourselin,et al.  Automatic Brain Tumor Segmentation Using Cascaded Anisotropic Convolutional Neural Networks , 2017, BrainLes@MICCAI.

[52]  Yair Lotan,et al.  Treatment of Non‐Metastatic Muscle‐Invasive Bladder Cancer: AUA/ASCO/ASTRO/SUO Guideline , 2017, The Journal of urology.

[53]  Sébastien Ourselin,et al.  Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations , 2017, DLMIA/ML-CDS@MICCAI.

[54]  Deniz Erdogmus,et al.  Tversky Loss Function for Image Segmentation Using 3D Fully Convolutional Deep Networks , 2017, MLMI@MICCAI.

[55]  Lubomir M. Hadjiiski,et al.  Segmentation of inner and outer bladder wall using deep-learning convolutional neural network in CT urography , 2017, Medical Imaging.

[56]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[57]  Konstantinos Kamnitsas,et al.  DeepCut: Object Segmentation From Bounding Box Annotations Using Convolutional Neural Networks , 2016, IEEE Transactions on Medical Imaging.

[58]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[59]  Christopher Joseph Pal,et al.  Brain tumor segmentation with Deep Neural Networks , 2015, Medical Image Anal..

[60]  Lubomir M. Hadjiiski,et al.  Bladder Cancer Segmentation in CT for Treatment Response Assessment: Application of Deep-Learning Convolution Neural Network—A Pilot Study , 2016, Tomography.

[61]  M. Milowsky,et al.  Guideline on Muscle-Invasive and Metastatic Bladder Cancer (European Association of Urology Guideline): American Society of Clinical Oncology Clinical Practice Guideline Endorsement. , 2016, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[62]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[63]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[64]  Jian Sun,et al.  ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[65]  Yaozong Gao,et al.  Fully convolutional networks for multi-modality isointense infant brain image segmentation , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[66]  Lubomir M. Hadjiiski,et al.  Urinary bladder segmentation in CT urography using deep-learning convolutional neural network and level sets. , 2016, Medical physics.

[67]  Max A. Viergever,et al.  Automatic Segmentation of MR Brain Images With a Convolutional Neural Network , 2016, IEEE Transactions on Medical Imaging.

[68]  Iasonas Kokkinos,et al.  Sub-cortical brain structure segmentation using F-CNN'S , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[69]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[70]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[71]  Shuiwang Ji,et al.  Deep convolutional neural networks for multi-modality isointense infant brain image segmentation , 2015, NeuroImage.

[72]  Giovanni Montana,et al.  Deep neural networks for anatomical brain segmentation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[73]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[74]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[75]  Xi Zhang,et al.  3D detection and extraction of bladder tumors via MR virtual cystoscopy , 2015, International Journal of Computer Assisted Radiology and Surgery.

[76]  Xuelong Li,et al.  Adaptive Shape Prior Constrained Level Sets for Bladder MR Image Segmentation , 2014, IEEE Journal of Biomedical and Health Informatics.

[77]  W. Horninger,et al.  Perforation during TUR of bladder tumours influences the natural history of superficial bladder cancer , 2014, World Journal of Urology.

[78]  Zhengrong Liang,et al.  A unified EM approach to bladder wall segmentation with coupled level-set constraints , 2013, Medical Image Anal..

[79]  Christian Igel,et al.  Deep Feature Learning for Knee Cartilage Segmentation Using a Triplanar Convolutional Neural Network , 2013, MICCAI.

[80]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[81]  Zhengrong Liang,et al.  An Adaptive Window-Setting Scheme for Segmentation of Bladder Tumor Surface via MR Cystography , 2012, IEEE Transactions on Information Technology in Biomedicine.

[82]  Arjan Bel,et al.  Automatic bladder segmentation on CBCT for multiple plan ART of bladder cancer using a patient-specific bladder model , 2012, Physics in medicine and biology.

[83]  H. Mostafid,et al.  Measuring and improving the quality of transurethral resection for bladder tumour (TURBT) , 2012, BJU international.

[84]  Vladlen Koltun,et al.  Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials , 2011, NIPS.

[85]  Ke Wu,et al.  Bladder segmentation in MRI images using active region growing model , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[86]  Ben Glocker,et al.  Deformable medical image registration: setting the state of the art with discrete methods. , 2011, Annual review of biomedical engineering.

[87]  Michael Brady,et al.  Segmentation of the bladder wall using coupled level set methods , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[88]  João Manuel R. S. Tavares,et al.  Novel Approach to Segment the Inner and Outer Boundaries of the Bladder Wall in T2-Weighted Magnetic Resonance Images , 2011, Annals of Biomedical Engineering.

[89]  Zhengrong Liang,et al.  A Coupled Level Set Framework for Bladder Wall Segmentation With Application to MR Cystography , 2010, IEEE Transactions on Medical Imaging.

[90]  J. Gschwend,et al.  An updated critical analysis of the treatment strategy for newly diagnosed high-grade T1 (previously T1G3) bladder cancer. , 2010, European urology.

[91]  Axel Hoos,et al.  Guidelines for the Evaluation of Immune Therapy Activity in Solid Tumors: Immune-Related Response Criteria , 2009, Clinical Cancer Research.

[92]  L. Schwartz,et al.  New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1). , 2009, European journal of cancer.

[93]  Zhengrong Liang,et al.  Segmentation of multispectral bladder MR images with inhomogeneity correction for virtual cystoscopy , 2008, SPIE Medical Imaging.

[94]  Yair Lotan,et al.  Nomogram for predicting disease recurrence after radical cystectomy for transitional cell carcinoma of the bladder. , 2006, The Journal of urology.

[95]  J Alfred Witjes,et al.  Predicting recurrence and progression in individual patients with stage Ta T1 bladder cancer using EORTC risk tables: a combined analysis of 2596 patients from seven EORTC trials. , 2006, European urology.

[96]  D. Bluemke,et al.  Dynamic MRI of bladder cancer: evaluation of staging accuracy. , 2005, AJR. American journal of roentgenology.

[97]  J E Husband,et al.  Evaluation of the response to treatment of solid tumours – a consensus statement of the International Cancer Imaging Society , 2004, British Journal of Cancer.

[98]  Zhengrong Liang,et al.  A new partial volume segmentation approach to extract bladder wall for computer-aided detection in virtual cystoscopy , 2004, SPIE Medical Imaging.

[99]  M. Peyromaure,et al.  The value of a second transurethral resection in evaluating patients with bladder tumours. , 2003, European urology.

[100]  J. Sethian,et al.  FRONTS PROPAGATING WITH CURVATURE DEPENDENT SPEED: ALGORITHMS BASED ON HAMILTON-JACOB1 FORMULATIONS , 2003 .

[101]  Keith J. Burnham,et al.  Automatic segmentation of clinical structures for RTP: Evaluation of a morphological approach , 2001 .

[102]  M. Palmer,et al.  WHO Handbook for Reporting Results of Cancer Treatment , 1982, British Journal of Cancer.

[103]  H. C. Jones,et al.  The treatment of tumours of the bladder by transurethral resection. , 1962, British journal of urology.