Evolutionary Multitasking AUC Optimization [Research Frontier]

Learning to optimize the area under the receiver operating characteristics curve (AUC) performance for imbalanced data has attracted much attention in recent years. Although there have been several methods of AUC optimization, scaling up AUC optimization is still an open issue due to its pairwise learning style. Maximizing AUC in the large-scale dataset can be considered as a non-convex and expensive problem. Inspired by the characteristic of pairwise learning, the cheap AUC optimization task with a small-scale dataset sampled from the large-scale dataset is constructed to promote the AUC accuracy of the original, large-scale, and expensive AUC optimization task. This paper develops an evolutionary multitasking framework (termed EMTAUC) to make full use of information among the constructed cheap and expensive tasks to obtain higher performance. In EMTAUC, one mission is to optimize AUC from the sampled dataset, and the other is to maximize AUC from the original dataset. Moreover, due to the cheap task containing limited knowledge, a strategy for dynamically adjusting the data structure of inexpensive tasks is proposed to introduce more knowledge into the multitasking AUC optimization environment. The performance of the proposed method is evaluated on a series of binary classification datasets. The experimental results demonstrate that EMTAUC is highly competitive to single task methods and online methods. Supplementary materials and source code implementation of EMTAUC can be accessed at https://github.com/xiaofangxd/EMTAUC.

[1]  G. Lugosi,et al.  Ranking and empirical minimization of U-statistics , 2006, math/0603123.

[2]  Lei Zhang,et al.  An adaptive mini-batch stochastic gradient method for AUC maximization , 2018, Neurocomputing.

[3]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Tutorial , 2016, ArXiv.

[4]  Yi Ding,et al.  An Adaptive Gradient Method for Online AUC Maximization , 2015, AAAI.

[5]  Xin Yao,et al.  Multiobjective genetic programming for maximizing ROC performance , 2014, Neurocomputing.

[6]  Koby Crammer,et al.  Online Passive-Aggressive Algorithms , 2003, J. Mach. Learn. Res..

[7]  Tianyou Chai,et al.  Generalized Multitasking for Evolutionary Optimization of Expensive Problems , 2019, IEEE Transactions on Evolutionary Computation.

[8]  Jing Liu,et al.  Evolutionary Multitasking Multilayer Network Reconstruction , 2021, IEEE Transactions on Cybernetics.

[9]  Kay Chen Tan,et al.  Multiobjective Multifactorial Optimization in Evolutionary Multitasking , 2017, IEEE Transactions on Cybernetics.

[10]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[11]  Prateek Jain,et al.  On the Generalization Ability of Online Learning Algorithms for Pairwise Loss Functions , 2013, ICML.

[12]  Kai Wu,et al.  Learning large-scale fuzzy cognitive maps using an evolutionary many-task algorithm , 2021, Appl. Soft Comput..

[13]  Ke Tang,et al.  Convex hull-based multi-objective evolutionary computation for maximizing receiver operating characteristics performance , 2015, Memetic Computing.

[14]  Chih-Jen Lin,et al.  Large-Scale Linear RankSVM , 2014, Neural Computation.

[15]  Rong Jin,et al.  Online AUC Maximization , 2011, ICML.

[16]  Licheng Jiao,et al.  Transfer stacking from low-to high-fidelity: A surrogate-assisted bi-fidelity evolutionary algorithm , 2020, Appl. Soft Comput..

[17]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[18]  Yew-Soon Ong,et al.  Evolutionary Optimization of Expensive Multiobjective Problems With Co-Sub-Pareto Front Gaussian Process Surrogates , 2019, IEEE Transactions on Cybernetics.

[19]  Kay Chen Tan,et al.  An Effective Knowledge Transfer Approach for Multiobjective Multitasking Optimization , 2020, IEEE Transactions on Cybernetics.

[20]  Stéphan Clémençon,et al.  Ranking the Best Instances , 2006, J. Mach. Learn. Res..

[21]  Jaume Bacardit,et al.  Speeding up the evaluation of evolutionary learning systems using GPGPUs , 2010, GECCO '10.

[22]  Abhishek Gupta,et al.  Multifactorial Evolutionary Algorithm With Online Transfer Parameter Estimation: MFEA-II , 2020, IEEE Transactions on Evolutionary Computation.

[23]  Liang Feng,et al.  An Adaptive Archive-Based Evolutionary Framework for Many-Task Optimization , 2020, IEEE Transactions on Emerging Topics in Computational Intelligence.

[24]  Thorsten Joachims,et al.  A support vector method for multivariate performance measures , 2005, ICML.

[25]  Jaume Bacardit,et al.  Large-scale experimental evaluation of GPU strategies for evolutionary machine learning , 2016, Inf. Sci..

[26]  Yew-Soon Ong,et al.  Multifactorial Evolution: Toward Evolutionary Multitasking , 2016, IEEE Transactions on Evolutionary Computation.

[27]  Lei Zhou,et al.  Evolutionary Multitasking via Explicit Autoencoding , 2019, IEEE Transactions on Cybernetics.

[28]  Abhishek Gupta,et al.  Cognizant Multitasking in Multiobjective Multifactorial Evolution: MO-MFEA-II , 2020, IEEE Transactions on Cybernetics.

[29]  Chi-Keong Goh,et al.  Multiproblem Surrogates: Transfer Evolutionary Multiobjective Optimization of Computationally Expensive Problems , 2019, IEEE Transactions on Evolutionary Computation.

[30]  Gary G. Yen,et al.  Transfer Learning-Based Dynamic Multiobjective Optimization Algorithms , 2016, IEEE Transactions on Evolutionary Computation.

[31]  Tie-Yan Liu,et al.  LightGBM: A Highly Efficient Gradient Boosting Decision Tree , 2017, NIPS.

[32]  Maoguo Gong,et al.  Self-Regulated Evolutionary Multitask Optimization , 2020, IEEE Transactions on Evolutionary Computation.

[33]  Zhi-Hua Zhou,et al.  One-Pass AUC Optimization , 2013, ICML.

[34]  Fang Liu,et al.  3D fast convex-hull-based evolutionary multiobjective optimization algorithm , 2018, Appl. Soft Comput..

[35]  Licheng Jiao,et al.  Multiobjective optimization of classifiers by means of 3D convex-hull-based evolutionary algorithms , 2014, Inf. Sci..

[36]  Xin Yao,et al.  Convex Hull-Based Multiobjective Genetic Programming for Maximizing Receiver Operating Characteristic Performance , 2015, IEEE Transactions on Evolutionary Computation.

[37]  Martin V. Butz,et al.  Speeding-Up Pittsburgh Learning Classifier Systems: Modeling Time and Accuracy , 2004, PPSN.

[38]  Avishek Saha,et al.  MBA: Mini-Batch AUC Optimization , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[39]  Zefeng Chen,et al.  Evolutionary Machine Learning With Minions: A Case Study in Feature Selection , 2021, IEEE Transactions on Evolutionary Computation.

[40]  Kai Wu,et al.  Solving Multitask Optimization Problems With Adaptive Knowledge Transfer via Anomaly Detection , 2022, IEEE Transactions on Evolutionary Computation.

[41]  Jun Zhang,et al.  Evolutionary Computation Meets Machine Learning: A Survey , 2011, IEEE Computational Intelligence Magazine.

[42]  John Doherty,et al.  A Generic Test Suite for Evolutionary Multifidelity Optimization , 2018, IEEE Transactions on Evolutionary Computation.

[43]  Chuan-Kang Ting,et al.  Evolutionary Manytasking Optimization Based on Symbiosis in Biocoenosis , 2019, AAAI.

[44]  Lei Zhang,et al.  A multi-level knee point based multi-objective evolutionary algorithm for AUC maximization , 2019, Memetic Comput..

[45]  Jing Liu,et al.  Evolutionary multitasking fuzzy cognitive map learning , 2020, Knowl. Based Syst..

[46]  Abhishek Gupta,et al.  Explicit Evolutionary Multitasking for Combinatorial Optimization: A Case Study on Capacitated Vehicle Routing Problem , 2020, IEEE Transactions on Cybernetics.

[47]  Mengjie Zhang,et al.  Multiobjective Multitasking Optimization Based on Incremental Learning , 2020, IEEE Transactions on Evolutionary Computation.

[48]  Xingyi Zhang,et al.  Multi-objective evolutionary algorithm for optimizing the partial area under the ROC curve , 2019, Knowledge-Based Systems.

[49]  Kay Chen Tan,et al.  Evolutionary Transfer Optimization - A New Frontier in Evolutionary Computation Research , 2021, IEEE Computational Intelligence Magazine.

[50]  Wentong Cai,et al.  Multifactorial Genetic Programming for Symbolic Regression Problems , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[51]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[52]  Jaume Bacardit,et al.  Automatic Tuning of Rule-Based Evolutionary Machine Learning via Problem Structure Identification , 2020, IEEE Computational Intelligence Magazine.

[53]  Jasper Snoek,et al.  Multi-Task Bayesian Optimization , 2013, NIPS.

[54]  Tianyou Chai,et al.  Offline Data-Driven Multiobjective Optimization: Knowledge Transfer Between Surrogates and Generation of Final Solutions , 2020, IEEE Transactions on Evolutionary Computation.

[55]  Jaume Bacardit Peñarroya Pittsburgh genetic-based machine learning in the data mining era: representations, generalization, and run-time , 2004 .