Md-knn: An Instance-based Approach for Multi-Dimensional Classification

Multi-dimensional classification (MDC) deals with the problem where each instance is associated with multiple class variables, each of which corresponds to a specific class space. One of the mainstream solutions for MDC is to adapt traditional machine learning techniques to deal with MDC data. In this paper, a first attempt towards adapting instance-based techniques for MDC is investigated, and a new approach named Md-knn is proposed. Specifically, Md-knn identifies unseen instance's $k$ nearest neighbors and obtains its corresponding $k\text{NN}$ counting statistics for each class space, based on which maximum a posteriori (MAP) inference is made for each pair of class spaces. After that, the class label w.r.t. each class space is determined by synergizing predictions from the learned classifiers via consulting empirical $k\text{NN}$ accuracy. Comparative studies over ten benchmark data sets clearly validate Md-knn's effectiveness.

[1]  Luca Martino,et al.  Efficient monte carlo methods for multi-dimensional learning with classifier chains , 2012, Pattern Recognit..

[2]  Hagit Shatkay,et al.  Multi-dimensional classification of biomedical text: Toward automated, practical provision of high-utility text to diverse users , 2008, Bioinform..

[3]  Zhi-Hua Zhou,et al.  ML-KNN: A lazy learning approach to multi-label learning , 2007, Pattern Recognit..

[4]  Min-Ling Zhang,et al.  Maximum Margin Multi-Dimensional Classification , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[5]  Janneke H. Bolt,et al.  Balanced sensitivity functions for tuning multi-dimensional Bayesian network classifiers , 2017, Int. J. Approx. Reason..

[6]  Concha Bielza,et al.  International Journal of Approximate Reasoning Tractability of most probable explanations in multidimensional Bayesian network classifiers ✩ , 2022 .

[7]  Xin Geng,et al.  Binary relevance for multi-label learning: an overview , 2018, Frontiers of Computer Science.

[8]  Min-Ling Zhang,et al.  A Review on Multi-Label Learning Algorithms , 2014, IEEE Transactions on Knowledge and Data Engineering.

[9]  Concha Bielza,et al.  Multi-Dimensional Classification with Super-Classes , 2014, IEEE Transactions on Knowledge and Data Engineering.

[10]  Bartosz Krawczyk,et al.  Multi-Label Punitive kNN with Self-Adjusting Memory for Drifting Data Streams , 2019, ACM Trans. Knowl. Discov. Data.

[11]  Vladimir Pavlovic,et al.  Copula Ordinal Regression for Joint Estimation of Facial Action Unit Intensity , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  C. Bielza,et al.  PREDICTING THE EQ-5D FROM THE PARKINSON'S DISEASE QUESTIONNAIRE PDQ-8 USING MULTI-DIMENSIONAL BAYESIAN NETWORK CLASSIFIERS , 2014 .

[13]  Ivor W. Tsang,et al.  Survey on Multi-Output Learning , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Min-Ling Zhang,et al.  Multi-dimensional classification via kNN feature augmentation , 2020, Pattern Recognit..

[15]  Sebastián Ventura,et al.  A Tutorial on Multilabel Learning , 2015, ACM Comput. Surv..

[16]  Sanyang Liu,et al.  A hybrid method for learning multi-dimensional Bayesian network classifiers based on an optimization model , 2015, Applied Intelligence.

[17]  Concha Bielza,et al.  Predicting human immunodeficiency virus inhibitors using multi-dimensional Bayesian network classifiers , 2013, Artif. Intell. Medicine.

[18]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[19]  Chen Chen,et al.  Incorporating Label Embedding and Feature Augmentation for Multi-Dimensional Classification , 2020, AAAI.

[20]  Songcan Chen,et al.  A convex formulation for multiple ordinal output classification , 2019, Pattern Recognit..

[21]  Iñaki Inza,et al.  Assisting in search heuristics selection through multidimensional supervised classification: A case study on software testing , 2014, Inf. Sci..

[22]  José Antonio Lozano,et al.  Using Multidimensional Bayesian Network Classifiers to Assist the Treatment of Multiple Sclerosis , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[23]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[24]  Linda C. van der Gaag,et al.  Multi-dimensional Bayesian Network Classifiers , 2006, Probabilistic Graphical Models.

[25]  Min-Ling Zhang,et al.  Multi-Dimensional Classification via kNN Feature Augmentation , 2019, AAAI.

[26]  María Concepción Bielza Lozoya,et al.  Multidimensional classifiers for neuroanatomical data , 2015, ICML 2015.

[27]  Hiroaki Harai,et al.  Multi-Target Classification Based Automatic Virtual Resource Allocation Scheme , 2019, IEICE Trans. Inf. Syst..

[28]  Min-Ling Zhang,et al.  A k-Nearest Neighbor Based Multi-Instance Multi-Label Learning Algorithm , 2010, 2010 22nd IEEE International Conference on Tools with Artificial Intelligence.

[29]  Min-Ling Zhang,et al.  Multi-dimensional classification via stacked dependency exploitation , 2020, Science China Information Sciences.

[30]  Concha Bielza,et al.  Bayesian Chain Classifiers for Multidimensional Classification , 2011, IJCAI.

[31]  Songcan Chen,et al.  Multi-dimensional classification via a metric approach , 2018, Neurocomputing.

[32]  Patrick Thiam,et al.  A k -Nearest Neighbor Based Algorithm for Multi-Instance Multi-Label Active Learning , 2018, ANNPR.

[33]  Concha Bielza,et al.  Multi-dimensional classification of GABAergic interneurons with Bayesian network-modeled label uncertainty , 2014, Front. Comput. Neurosci..

[34]  Eyke Hüllermeier,et al.  Combining instance-based learning and logistic regression for multilabel classification , 2009, Machine Learning.

[35]  Concha Bielza,et al.  Multi-dimensional classification with Bayesian networks , 2011, Int. J. Approx. Reason..

[36]  Verayuth Lertnattee,et al.  Multi-Dimensional Text Classification , 2002, COLING.

[37]  José Antonio Lozano,et al.  Multi-Objective Learning of Multi-Dimensional Bayesian Classifiers , 2008, 2008 Eighth International Conference on Hybrid Intelligent Systems.

[38]  Linda C. van der Gaag,et al.  Inference and Learning in Multi-dimensional Bayesian Network Classifiers , 2007, ECSQARU.

[39]  Concha Bielza,et al.  Multi-dimensional Bayesian Network Classifier Trees , 2018, IDEAL.