Compositional Training for End-to-End Deep AUC Maximization

cross-entropy (CE) algorithm. The proposed enhances the capabil-ities of both feature learning and robust classifier learning by taking a gradient step for the CE loss and for the AUC loss in a systematic way. We conduct extensive empirical studies on imbalanced benchmark and medical image datasets, which unanimously verify the effectiveness of the proposed method. Our results show that the compositional

[1]  Nitesh V. Chawla,et al.  DeepSMOTE: Fusing Deep Learning and SMOTE for Imbalanced Data , 2021, IEEE Transactions on Neural Networks and Learning Systems.

[2]  Shuiwang Ji,et al.  Advanced Graph and Sequence Neural Networks for Molecular Property Prediction and Drug Discovery. , 2020, Bioinformatics.

[3]  Deep Vision for Breast Cancer Classification and Segmentation , 2021, Cancers.

[4]  Quoc V. Le,et al.  EfficientNetV2: Smaller Models and Faster Training , 2021, ICML.

[5]  Tianbao Yang,et al.  Federated Deep AUC Maximization for Heterogeneous Data with a Constant Communication Complexity , 2021, ICML.

[6]  Stella X. Yu,et al.  Long-tailed Recognition by Routing Diverse Distribution-Aware Experts , 2020, ICLR.

[7]  George Shih,et al.  A patient-centric dataset of images and metadata for identifying melanomas using clinical context , 2020, Scientific Data.

[8]  Ankit Singh Rawat,et al.  Long-tail learning via logit adjustment , 2020, ICLR.

[9]  Tianbao Yang,et al.  On Stochastic Moving-Average Estimators for Non-Convex Optimization , 2021, ArXiv.

[10]  Seiichi Uchida,et al.  Top-rank convolutional neural network and its application to medical image-based diagnosis , 2021, Pattern Recognit..

[11]  Attentional Biased Stochastic Gradient for Imbalanced Classification , 2020, ArXiv.

[12]  Milan Sonka,et al.  Robust Deep AUC Maximization: A New Surrogate Loss and Empirical Studies on Medical Image Classification , 2020, ArXiv.

[13]  Zhi Xu,et al.  Rethinking the Value of Labels for Improving Class-Imbalanced Learning , 2020, NeurIPS.

[14]  Tianbao Yang,et al.  Fast Objective and Duality Gap Convergence for Non-convex Strongly-concave Min-max Problems , 2020, ArXiv.

[15]  Yi Yang,et al.  Inflated Episodic Memory With Region Self-Attention for Long-Tailed Visual Recognition , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Ming-Hsuan Yang,et al.  Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition From a Domain Adaptation Perspective , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Guiguang Ding,et al.  Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classification , 2020, ECCV.

[18]  Xiu-Shen Wei,et al.  BBN: Bilateral-Branch Network With Cumulative Learning for Long-Tailed Visual Recognition , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Saining Xie,et al.  Decoupling Representation and Classifier for Long-Tailed Recognition , 2019, ICLR.

[20]  Mingrui Liu,et al.  Stochastic AUC Maximization with Deep Neural Networks , 2019, ICLR.

[21]  Aryan Mokhtari,et al.  On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithms , 2019, AISTATS.

[22]  Michael I. Jordan,et al.  On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems , 2019, ICML.

[23]  Saeed Ghadimi,et al.  A Single Timescale Stochastic Approximation Method for Nested Stochastic Optimization , 2018, SIAM J. Optim..

[24]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Siwei Lyu,et al.  Stochastic AUC Optimization Algorithms With Linear Convergence , 2019, Front. Appl. Math. Stat..

[26]  Colin Wei,et al.  Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss , 2019, NeurIPS.

[27]  Stella X. Yu,et al.  Large-Scale Long-Tailed Recognition in an Open World , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Taghi M. Khoshgoftaar,et al.  Survey on deep learning with class imbalance , 2019, J. Big Data.

[29]  Xu Sun,et al.  Adaptive Gradient Methods with Dynamic Bound of Learning Rate , 2019, ICLR.

[30]  Yifan Yu,et al.  CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison , 2019, AAAI.

[31]  Yang Song,et al.  Class-Balanced Loss Based on Effective Number of Samples , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Siwei Lyu,et al.  Stochastic Proximal Algorithms for AUC Maximization , 2018, ICML.

[33]  Mingrui Liu,et al.  Fast Stochastic AUC Maximization with O(1/n)-Convergence Rate , 2018, ICML.

[34]  Max Welling,et al.  Rotation Equivariant CNNs for Digital Pathology , 2018, MICCAI.

[35]  Bin Yang,et al.  Learning to Reweight Examples for Robust Deep Learning , 2018, ICML.

[36]  Sashank J. Reddi,et al.  On the Convergence of Adam and Beyond , 2018, ICLR.

[37]  Saeed Ghadimi,et al.  Approximation Methods for Bilevel Programming , 2018, 1802.02246.

[38]  Mohammed Bennamoun,et al.  Cost-Sensitive Learning of Deep Feature Representations From Imbalanced Data , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[39]  Daniel L Rubin,et al.  A curated mammography data set for use in computer-aided detection and diagnosis research , 2017, Scientific Data.

[40]  Andrew H. Beck,et al.  Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer , 2017, JAMA.

[41]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[42]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Mengdi Wang,et al.  Stochastic compositional gradient descent: algorithms for minimizing compositions of expected-value functions , 2014, Mathematical Programming.

[44]  Siwei Lyu,et al.  Stochastic Online AUC Maximization , 2016, NIPS.

[45]  Hansang Lee,et al.  Plankton classification on imbalanced large scale database via convolutional neural networks with transfer learning , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[46]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[47]  Zhi-Hua Zhou,et al.  On the Consistency of AUC Pairwise Optimization , 2012, IJCAI.

[48]  David Masko,et al.  The Impact of Imbalanced Training Data for Convolutional Neural Networks , 2015 .

[49]  Prateek Jain,et al.  Online and Stochastic Gradient Methods for Non-decomposable Loss Functions , 2014, NIPS.

[50]  Prateek Jain,et al.  On the Generalization Ability of Online Learning Algorithms for Pairwise Loss Functions , 2013, ICML.

[51]  Zhi-Hua Zhou,et al.  One-Pass AUC Optimization , 2013, ICML.

[52]  Xinhua Zhang,et al.  Smoothing multivariate performance measures , 2011, J. Mach. Learn. Res..

[53]  Rong Jin,et al.  Online AUC Maximization , 2011, ICML.

[54]  Richard H. Moore,et al.  THE DIGITAL DATABASE FOR SCREENING MAMMOGRAPHY , 2007 .

[55]  Thorsten Joachims,et al.  A support vector method for multivariate performance measures , 2005, ICML.

[56]  Bhavani Raskutti,et al.  Optimising area under the ROC curve using gradient descent , 2004, ICML.

[57]  Michael C. Mozer,et al.  Optimizing Classifier Performance via an Approximation to the Wilcoxon-Mann-Whitney Statistic , 2003, ICML.

[58]  Peter A. Flach,et al.  Learning Decision Trees Using the Area Under the ROC Curve , 2002, ICML.

[59]  Thore Graepel,et al.  Large Margin Rank Boundaries for Ordinal Regression , 2000 .

[60]  Yoram Singer,et al.  An Efficient Boosting Algorithm for Combining Preferences by , 2013 .

[61]  Richard H. Moore,et al.  Current Status of the Digital Database for Screening Mammography , 1998, Digital Mammography / IWDM.