Active Image Synthesis for Efficient Labeling.

The great success achieved by deep neural networks attracts increasing attention from the manufacturing and healthcare communities. However, the limited availability of data and high costs of data collection are the major challenges for the applications in those fields. We propose in this work AISEL, an active image synthesis method for efficient labeling, to improve the performance of the small-data learning tasks. Specifically, a complementary AISEL dataset is generated, with labels actively acquired via a physics-based method to incorporate underlining physical knowledge at hand. An important component of our AISEL method is the bidirectional generative invertible network (GIN), which can extract interpretable features from the training images and generate physically meaningful virtual images. Our AISEL method then efficiently samples virtual images not only further exploits the uncertain regions but also explores the entire image space. We then discuss the interpretability of GIN both theoretically and experimentally, demonstrating clear visual improvements over the benchmarks. Finally, we demonstrate the effectiveness of our AISEL framework on aortic stenosis application, in which our method lowers the labeling cost by 90% while achieving a 15% improvement in prediction accuracy.

[1]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[2]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[3]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[4]  Prafulla Dhariwal,et al.  Glow: Generative Flow with Invertible 1x1 Convolutions , 2018, NeurIPS.

[5]  Z. Qian,et al.  Quantitative Prediction of Paravalvular Leak in Transcatheter Aortic Valve Replacement Based on Tissue-Mimicking 3D Printing. , 2017, JACC Cardiovascular Imaging.

[6]  Nikos Komodakis,et al.  Wide Residual Networks , 2016, BMVC.

[7]  V. Roshan Joseph,et al.  Support points , 2016, The Annals of Statistics.

[8]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[9]  Dazhong Wu,et al.  Deep learning for smart manufacturing: Methods and applications , 2018, Journal of Manufacturing Systems.

[10]  Stefano Ermon,et al.  Towards Deeper Understanding of Variational Autoencoding Models , 2017, ArXiv.

[11]  Nicolas Courty,et al.  Distance Measure Machines , 2018, 1803.00250.

[12]  Zoubin Ghahramani,et al.  Deep Bayesian Active Learning with Image Data , 2017, ICML.

[13]  B. J. Winer Statistical Principles in Experimental Design , 1992 .

[14]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[15]  V. R. Joseph,et al.  Projected support points, with application to optimal MCMC reduction , 2017 .

[16]  Kan Wang,et al.  An efficient statistical approach to design 3D-printed metamaterials for mimicking mechanical properties of soft biological tissues , 2018, Additive Manufacturing.

[17]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Sylvia Richardson,et al.  Markov Chain Monte Carlo in Practice , 1997 .

[19]  Astrid. Fagraeus,et al.  Antibody Production in relation to the Development of Plasma Cells. In vivo and in vitro Experiments. , 1948 .

[20]  Thomas J. Santner,et al.  The Design and Analysis of Computer Experiments , 2003, Springer Series in Statistics.

[21]  Paris Perdikaris,et al.  Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations , 2017, ArXiv.

[22]  L. Chitty,et al.  Fetal Size and Dating: Charts Recommended for Clinical Obstetric Practice , 2009 .

[23]  Tirthankar Dasgupta,et al.  Sequential Exploration of Complex Surfaces Using Minimum Energy Designs , 2015, Technometrics.

[24]  Yann LeCun,et al.  Convolutional networks and applications in vision , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[25]  Ruslan Salakhutdinov,et al.  How Many Samples are Needed to Estimate a Convolutional or Recurrent Neural Network , 2018 .

[26]  Frédéric Precioso,et al.  Adversarial Active Learning for Deep Networks: a Margin Based Approach , 2018, ArXiv.

[27]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[28]  V. R. Joseph,et al.  An Efficient Surrogate Model for Emulation and Physics Extraction of Large Eddy Simulations , 2016, Journal of the American Statistical Association.

[29]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[30]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[31]  Michał Grochowski,et al.  Data augmentation for improving deep learning in image classification problem , 2018, 2018 International Interdisciplinary PhD Workshop (IIPhDW).

[32]  Nitesh V. Chawla,et al.  SPECIAL ISSUE ON LEARNING FROM IMBALANCED DATA SETS , 2004 .

[33]  C. Villani Optimal Transport: Old and New , 2008 .

[34]  G. Matheron Principles of geostatistics , 1963 .

[35]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Ben Wang,et al.  Generative Invertible Networks (GIN): Pathophysiology-Interpretable Feature Mapping and Virtual Patient Generation , 2018, MICCAI.

[37]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[38]  S. Troian,et al.  A general boundary condition for liquid flow at solid surfaces , 1997, Nature.

[39]  Jonathon Shlens,et al.  Conditional Image Synthesis with Auxiliary Classifier GANs , 2016, ICML.

[40]  Huan Yan,et al.  Engineering-Driven Statistical Adjustment and Calibration , 2015, Technometrics.

[41]  Luis Perez,et al.  The Effectiveness of Data Augmentation in Image Classification using Deep Learning , 2017, ArXiv.

[42]  Roland Vollgraf,et al.  Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms , 2017, ArXiv.

[43]  José Bento,et al.  Generative Adversarial Active Learning , 2017, ArXiv.

[44]  Hayit Greenspan,et al.  Synthetic data augmentation using GAN for improved liver lesion classification , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[45]  Hongyuan Zha,et al.  A Fast Proximal Point Method for Computing Wasserstein Distance , 2018 .

[46]  Mateu Sbert,et al.  Image Segmentation Using Excess Entropy , 2009, J. Signal Process. Syst..

[47]  George E. Karniadakis,et al.  Hidden physics models: Machine learning of nonlinear partial differential equations , 2017, J. Comput. Phys..

[48]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[49]  Haibo He,et al.  Learning from Imbalanced Data , 2009, IEEE Transactions on Knowledge and Data Engineering.

[50]  Trevor Darrell,et al.  Adversarial Feature Learning , 2016, ICLR.

[51]  Burr Settles,et al.  Active Learning , 2012, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[52]  Catherine M. Otto,et al.  Clinical Factors Associated With Calcific Aortic Valve Disease , 1997 .

[53]  Ling Shao,et al.  Transfer Learning for Visual Categorization: A Survey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[54]  S. Resnick A Probability Path , 1999 .

[55]  Andrew Zisserman,et al.  Return of the Devil in the Details: Delving Deep into Convolutional Nets , 2014, BMVC.

[56]  C. F. Jeff Wu,et al.  Experiments , 2021, Wiley Series in Probability and Statistics.

[57]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[58]  Quanming Yao,et al.  Few-shot Learning: A Survey , 2019, ArXiv.

[59]  Aaron C. Courville,et al.  Adversarially Learned Inference , 2016, ICLR.

[60]  A. Ziv,et al.  Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review , 2005, Medical teacher.

[61]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[62]  E. Nadaraya On Estimating Regression , 1964 .

[63]  Kaibo Liu,et al.  A Nonparametric Adaptive Sampling Strategy for Online Monitoring of Big Data Streams , 2018, Technometrics.

[64]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[65]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[66]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[67]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[68]  Sivaraman Balakrishnan,et al.  How Many Samples are Needed to Learn a Convolutional Neural Network? , 2018, NIPS 2018.