Learning Generic Lung Ultrasound Biomarkers for Decoupling Feature Extraction from Downstream Tasks

, Abstract. Contemporary artificial neural networks (ANN) are trained end-to-end, jointly learning both features and classifiers for the task of interest. Though enormously effective, this paradigm imposes signif-icant costs in assembling annotated task-specific datasets and training large-scale networks. We propose to decouple feature learning from downstream lung ultrasound tasks by introducing an auxiliary pre-task of visual biomarker classification. We demonstrate that one can learn an infor-mative, concise, and interpretable feature space from ultrasound videos by training models for predicting biomarker labels. Notably, biomarker feature extractors can be trained from data annotated with weak video-scale supervision. These features can be used by a variety of downstream Expert models targeted for diverse clinical tasks (Diagnosis, lung severity, S/F ratio). Crucially, task-specific expert models are comparable in accuracy to end-to-end models directly trained for such target tasks, while being significantly lower cost to train.

[1]  P. M. Gordaliza,et al.  Translational Lung Imaging Analysis Through Disentangled Representations , 2022, ArXiv.

[2]  Jeroen M. A. van der Burgt,et al.  Automatic deep learning-based consolidation/collapse classification in lung ultrasound images for COVID-19 induced pneumonia , 2022, Scientific Reports.

[3]  S. Aylward,et al.  Investigating training-test data splitting strategies for automated segmentation and scoring of COVID-19 lung ultrasound images. , 2021, The Journal of the Acoustical Society of America.

[4]  Edmund Y. Lam,et al.  Transfer Learning U-Net Deep Learning for Lung Ultrasound Segmentation , 2021, ArXiv.

[5]  Gautam Rajendrakumar Gare,et al.  Dense Pixel-Labeling For Reverse-Transfer And Diagnostic Learning On Lung Ultrasound For Covid-19 And Pneumonia Detection , 2021, 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI).

[6]  Wufeng Xue,et al.  Modality alignment contrastive learning for severity assessment of COVID-19 from lung ultrasound and clinical information , 2021, Medical Image Analysis.

[7]  Karsten M. Borgwardt,et al.  Accelerating detection of lung pathologies with explainable ultrasound image analysis , 2021 .

[8]  Cristiano Saltori,et al.  Deep Learning for Classification and Localization of COVID-19 Markers in Point-of-Care Lung Ultrasound , 2020, IEEE Transactions on Medical Imaging.

[9]  Hyo-Eun Kim,et al.  Changes in cancer detection and false-positive recall in mammography using artificial intelligence: a retrospective, multireader study. , 2020, The Lancet. Digital health.

[10]  Lamiaa A. Elrefaei,et al.  Deep Convolutional Neural Network-Based Approaches for Face Recognition , 2019, Applied Sciences.

[11]  D. Lichtenstein Current Misconceptions in Lung Ultrasound: A Short Guide for Experts. , 2019, Chest.

[12]  Chuang Gan,et al.  TSM: Temporal Shift Module for Efficient Video Understanding , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[13]  Marcus A. Badgeley,et al.  Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study , 2018, PLoS medicine.

[14]  Tao Tan,et al.  Optimize Transfer Learning for Lung Diseases in Bronchoscopy Using a New Concept: Sequential Fine-Tuning , 2018, IEEE Journal of Translational Engineering in Health and Medicine.

[15]  Zachary Chase Lipton The mythos of model interpretability , 2016, ACM Queue.

[16]  Andrew Zisserman,et al.  Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Subhransu Maji,et al.  Bilinear CNNs for Fine-grained Visual Recognition , 2015 .

[19]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[20]  M. Mostafizur Rahman,et al.  Addressing the Class Imbalance Problem in Medical Datasets , 2013 .

[21]  Léon Bottou,et al.  Large-Scale Machine Learning with Stochastic Gradient Descent , 2010, COMPSTAT.

[22]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..