OCEAn: Ordinal classification with an ensemble approach

Abstract Generally, classification problems catalog instances according to their target variable without considering the relation among the different labels. However, there are real problems in which the different values of the class are related to each other. Because of interest in this type of problem, several solutions have been proposed, such as cost-sensitive classifiers. Ensembles have proven to be very effective for classification tasks; however, as far as we know, there are no proposals that use a genetic-based methodology as the metaheuristic to create the models. In this paper, we present OCEAn, an ordinal classification algorithm based on an ensemble approach, which makes a final prediction according to a weighted vote system. This weighted voting takes into account weights obtained by a genetic algorithm that tries to minimize the cost of classification. To test the performance of this approach, we compared our proposal with ordinal classification algorithms in the literature and demonstrated that, indeed, our approach improves on previous results.

[1]  A. Greil,et al.  Concerns about treatment for infertility in a probability-based sample of US women , 2020, Journal of reproductive and infant psychology.

[2]  Yi-Ping Hung,et al.  Ordinal hyperplanes ranker with cost sensitivities for age estimation , 2011, CVPR 2011.

[3]  D. Opitz,et al.  Popular Ensemble Methods: An Empirical Study , 1999, J. Artif. Intell. Res..

[4]  D. Brodic,et al.  Methodology for the Evaluation of the Algorithms for Text Line Segmentation Based on Extended Binary Classification , 2011 .

[5]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[6]  Abdullah Alsaeedi,et al.  Software Defect Prediction Using Supervised Machine Learning and Ensemble Techniques: A Comparative Study , 2019, Journal of Software Engineering and Applications.

[7]  Wei Chu,et al.  Biomarker discovery in microarray gene expression data with Gaussian processes , 2005, Bioinform..

[8]  Ram Sarkar,et al.  Breast cancer detection using feature selection and active learning , 2017 .

[9]  Yiyu Yao,et al.  Artificial Intelligence Perspectives on Granular Computing , 2011 .

[10]  Han Liu,et al.  Nature-inspired framework of ensemble learning for collaborative classification in granular computing context , 2018, Granular Computing.

[11]  Araceli Sanchis,et al.  CCE: An ensemble architecture based on coupled ANN for solving multiclass problems , 2020, Inf. Fusion.

[12]  Francisco Herrera,et al.  Ensembles of cost-diverse Bayesian neural learners for imbalanced binary classification , 2020, Inf. Sci..

[13]  Pedro Antonio Gutiérrez,et al.  Cumulative link models for deep ordinal classification , 2019, Neurocomputing.

[14]  Nihal A. Siddiqui,et al.  Assessment of Occupational, Health and Safety Problems in the Automotive Industry and their Remedial Measures , 2014 .

[15]  Juan de Oña,et al.  A classification tree approach to identify key factors of transit service quality , 2012, Expert Syst. Appl..

[16]  Jing Cai,et al.  A Weighted Voting Classifier Based on Differential Evolution , 2014 .

[17]  Koby Crammer,et al.  Online Ranking by Projecting , 2005, Neural Computation.

[18]  Jan C. Bioch,et al.  A Decision Tree Algorithm for Ordinal Classification , 1999, IDA.

[19]  José Cristóbal Riquelme Santos,et al.  External clustering validity index based on chi-squared statistical test , 2019, Inf. Sci..

[20]  Francisco Herrera,et al.  Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power , 2010, Inf. Sci..

[21]  Willem Waegeman,et al.  An ensemble of Weighted Support Vector Machines for Ordinal Regression , 2007 .

[22]  Francisco Herrera,et al.  Deep Learning Hyper-parameter Tuning for Sentiment Analysis in Twitter based on Evolutionary Algorithms , 2019, 2019 Federated Conference on Computer Science and Information Systems (FedCSIS).

[23]  Pedro Antonio Gutiérrez,et al.  Ordinal Regression Methods: Survey and Experimental Study , 2016, IEEE Transactions on Knowledge and Data Engineering.

[24]  Klaus Hechenbichler,et al.  Weighted k-Nearest-Neighbor Techniques and Ordinal Classification , 2004 .

[25]  Masashi Sugiyama,et al.  Multi-Task Learning via Conic Programming , 2007, NIPS.

[26]  Eibe Frank,et al.  A Simple Approach to Ordinal Classification , 2001, ECML.

[27]  Wei Chu,et al.  Gaussian Processes for Ordinal Regression , 2005, J. Mach. Learn. Res..

[28]  Francesca Mangili,et al.  Should We Really Use Post-Hoc Tests Based on Mean-Ranks? , 2015, J. Mach. Learn. Res..

[29]  Lior Rokach,et al.  Ensemble learning: A survey , 2018, WIREs Data Mining Knowl. Discov..

[30]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[31]  Wei Chu,et al.  Support Vector Ordinal Regression , 2007, Neural Computation.

[32]  Sajjan G. Shiva,et al.  A Holistic Approach for Detecting DDoS Attacks by Using Ensemble Unsupervised Machine Learning , 2020, Advances in Intelligent Systems and Computing.

[33]  S. García,et al.  An Extension on "Statistical Comparisons of Classifiers over Multiple Data Sets" for all Pairwise Comparisons , 2008 .

[34]  Shifei Ding,et al.  A novel clustering ensemble model based on granular computing , 2021, Applied Intelligence.

[35]  Nathalie Japkowicz,et al.  Evaluation Methods for Ordinal Classification , 2009, Canadian Conference on AI.

[36]  Francisco Herrera,et al.  An overview of ensemble methods for binary classifiers in multi-class problems: Experimental study on one-vs-one and one-vs-all schemes , 2011, Pattern Recognit..

[37]  Shiliang Sun.  Traffic flow forecasting based on multitask ensemble learning , 2009, GEC '09.

[38]  W. Pedrycz,et al.  Information granules and their use in schemes of knowledge management , 2011, Sci. Iran..

[39]  Gang Hua,et al.  Ordinal Regression with Multiple Output CNN for Age Estimation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Derya Birant,et al.  EBOC: Ensemble-Based Ordinal Classification in Transportation , 2019, Journal of Advanced Transportation.

[41]  Mohamed A. Ismail,et al.  Multi-level gene/MiRNA feature selection using deep belief nets and active learning , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[42]  Wojciech Kotlowski,et al.  On Nonparametric Ordinal Classification with Monotonicity Constraints , 2013 .

[43]  Jaime S. Cardoso,et al.  Ordinal Data Classification Using Kernel Discriminant Analysis: A Comparison of Three Approaches , 2012, 2012 11th International Conference on Machine Learning and Applications.

[44]  Milosz Kadzinski,et al.  EMOSOR: Evolutionary multiple objective optimization guided by interactive stochastic ordinal regression , 2019, Comput. Oper. Res..

[45]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[46]  Qing Wang,et al.  Ensemble Learning Based on Multi-Task Class Labels , 2010, PAKDD.