Mapping to optimal regions; A new method for multi-class classification task to reduce complexity

Classification of data is an important problem which has attracted many researchers to introduce new approaches. In this paper, we propose Mapping to Optimal Regions (MOR) as a new method for multi-class classification task to reduce computational and memory complexities. It requires only one simple mapping from input space to optimal regions. The optimal domain is estimated using a multi objective cost function to increase the region size and the generalization ability of the mapping and to reduce the mapping error. Finally, the centers of optimal regions are determined with respect to the optimal size of the regions and the code assignment process which reduces the effect of inappropriate labeling. A Hierarchical version of MOR (HMOR) is presented for datasets with high number of classes or low dimensional feature spaces. By taking the advantage of MOR, the complexity reduces significantly in comparison to the other classifiers.

[1]  David G. Stork,et al.  Pattern Classification , 1973 .

[2]  Feiping Nie,et al.  Semi-supervised orthogonal discriminant analysis via label propagation , 2009, Pattern Recognit..

[3]  Hakan Erdogan,et al.  Max-Margin Stacking and Sparse Regularization for Linear Classifier Combination and Selection , 2011, ArXiv.

[4]  Baowei Fei,et al.  A modified fuzzy C-means classification method using a multiscale diffusion filtering scheme , 2009, Medical Image Anal..

[5]  J. Brian Gray,et al.  Introduction to Linear Regression Analysis , 2002, Technometrics.

[6]  Yuguo Wang,et al.  A Tree-Based Multi-class SVM Classifier for Digital Library Document , 2008, 2008 International Conference on MultiMedia and Information Technology.

[7]  R. K. Agrawal,et al.  Evaluation of Decision Tree SVM Framework Using Different Statistical Measures , 2009, 2009 International Conference on Advances in Recent Technologies in Communication and Computing.

[8]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[9]  Narendra Ahuja,et al.  Learning to Recognize 3D Objects with SNoW , 2000, ECCV.

[10]  S. Lewis,et al.  Regression analysis , 2007, Practical Neurology.

[11]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[12]  Francesco Camastra,et al.  LVQ-Based Hand Gesture Recognition Using a Data Glove , 2012, WIRN.

[13]  Tzong-Jer Chen,et al.  Fuzzy c-means clustering with spatial information for image segmentation , 2006, Comput. Medical Imaging Graph..

[14]  Alex Smola,et al.  Kernel methods in machine learning , 2007, math/0701907.

[15]  Lilong Cai,et al.  Adaptive-Fourier-Neural-Network-Based Control for a Class of Uncertain Nonlinear Systems , 2008, IEEE Transactions on Neural Networks.

[16]  Teuvo Kohonen,et al.  Things you haven't heard about the self-organizing map , 1993, IEEE International Conference on Neural Networks.

[17]  Hideyuki Imai,et al.  Probably correct k-nearest neighbor search in high dimensions , 2010, Pattern Recognit..

[18]  Lei Xu,et al.  A Trend on Regularization and Model Selection in Statistical Learning: A Bayesian Ying Yang Learning Perspective , 2007, Challenges for Computational Intelligence.

[19]  Song Ying,et al.  Face recognition based on multi-class SVM , 2009, 2009 Chinese Control and Decision Conference.

[20]  Kaare Brandt Petersen,et al.  The Matrix Cookbook , 2006 .

[21]  Xiaodan Wang,et al.  An Improved Algorithm for Decision-Tree-Based SVM , 2006, 2006 6th World Congress on Intelligent Control and Automation.

[22]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[23]  Urbano Nunes,et al.  Novel Maximum-Margin Training Algorithms for Supervised Neural Networks , 2010, IEEE Transactions on Neural Networks.

[24]  Albert Tarantola,et al.  Inverse problem theory - and methods for model parameter estimation , 2004 .

[25]  T. Murata,et al.  Advanced modularity-specialized label propagation algorithm for detecting communities in networks , 2009, 0910.1154.

[26]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[27]  Carl D. Meyer,et al.  Matrix Analysis and Applied Linear Algebra , 2000 .

[28]  Zbigniew W. Ras,et al.  Recognition of Instrument Timbres in Real Polytimbral Audio Recordings , 2010, ECML/PKDD.

[29]  Lennart Ljung,et al.  Modeling Of Dynamic Systems , 1994 .

[30]  Osamu Hasegawa,et al.  Associative Memory for Online Learning in Noisy Environments Using Self-Organizing Incremental Neural Network , 2009, IEEE Transactions on Neural Networks.

[31]  G. W. Stewart,et al.  Matrix algorithms , 1998 .

[32]  Masashi Sugiyama,et al.  Robust Label Propagation on Multiple Networks , 2009, IEEE Transactions on Neural Networks.

[33]  I. Burhan Türksen,et al.  Fuzzy functions with LSE , 2008, Appl. Soft Comput..

[34]  N. Draper,et al.  Applied Regression Analysis. , 1967 .

[35]  Teuvo Kohonen,et al.  Comparison of SOM Point Densities Based on Different Criteria , 1999, Neural Computation.

[36]  Mohammad Javad Yazdanpanah,et al.  Multiscale cancer modeling: In the line of fast simulation and chemotherapy , 2009, Math. Comput. Model..

[37]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[38]  Elizabeth A. Peck,et al.  Introduction to Linear Regression Analysis , 2001 .

[39]  Péter Komjáth,et al.  Problems And Theorems In Classical Set Theory , 2006 .

[40]  Robert Tibshirani,et al.  Discriminant Adaptive Nearest Neighbor Classification , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[41]  Michael T. Heath,et al.  Scientific Computing: An Introductory Survey , 1996 .

[42]  Gary Simon.  Applied Regression Modeling: A Business Approach , 2007 .

[43]  Helge J. Ritter,et al.  Large-scale data exploration with the hierarchically growing hyperbolic SOM , 2006, Neural Networks.

[44]  Shiliang Sun,et al.  Multiple-view multiple-learner active learning , 2010, Pattern Recognit..

[45]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[46]  Michael T. Heath,et al.  Scientific Computing , 2018 .

[47]  Jacek M. Zurada,et al.  Introduction to artificial neural systems , 1992 .

[48]  Jun Zhou,et al.  Mixing Linear SVMs for Nonlinear Classification , 2010, IEEE Transactions on Neural Networks.

[49]  Amaury Lendasse,et al.  X-SOM and L-SOM: A double classification approach for missing value imputation , 2010, Neurocomputing.

[50]  Hava T. Siegelmann,et al.  Support Vector Clustering , 2002, J. Mach. Learn. Res..

[51]  J. Stoer,et al.  Introduction to Numerical Analysis , 2002 .

[52]  Min Han,et al.  Efficient clustering of radial basis perceptron neural network for pattern recognition , 2004, Pattern Recognit..

[53]  Francesco Masulli,et al.  A survey of kernel and spectral methods for clustering , 2008, Pattern Recognit..

[54]  Martti Juhola,et al.  Self-Organising Maps in Document Classification: A Comparison with Six Machine Learning Methods , 2011, ICANNGA.

[55]  Qian Zhao,et al.  Joint Optimization of Feature Selection and Parameters for Multi-class SVM in Skin Symptomatic Recognition , 2009, 2009 International Conference on Artificial Intelligence and Computational Intelligence.

[56]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[57]  Martin A. Riedmiller,et al.  Incremental GRLVQ: Learning relevant features for 3D object recognition , 2008, Neurocomputing.

[58]  Cheng Wang,et al.  Adaptive binary tree for fast SVM multiclass classification , 2009, Neurocomputing.

[59]  Doheon Lee,et al.  Evaluation of the performance of clustering algorithms in kernel-induced feature space , 2005, Pattern Recognit..

[60]  Bernhard Schölkopf,et al.  Learning with kernels , 2001 .

[61]  Ashwani Kumar,et al.  Price forecasting using wavelet transform and LSE based mixed model in Australian electricity market , 2008 .

[62]  Yiguang Liu,et al.  A novel and quick SVM-based multi-class classifier , 2006, Pattern Recognit..

[63]  Cheong Hee Park,et al.  A SVM-based discretization method with application to associative classification , 2009, Expert Syst. Appl..

[64]  Piotr Synak,et al.  Extracting Emotions from Music Data , 2005, ISMIS.

[65]  Junfei Qiao,et al.  A Self-Organizing Fuzzy Neural Network Based on a Growing-and-Pruning Algorithm , 2010, IEEE Transactions on Fuzzy Systems.

[66]  Mohammad Javad Yazdanpanah,et al.  Image Compression using an Enhanced Self Organizing Map Algorithm with Vigilance Parameter , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[67]  Adnan Khashman,et al.  Modeling cognitive and emotional processes: A novel neural network architecture , 2010, Neural Networks.

[68]  Sheng Chen,et al.  A New RBF Neural Network With Boundary Value Constraints , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[69]  Jiansheng Chen,et al.  Piecewise linear aging function for facial age estimation , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[70]  Masaki Nakagawa,et al.  Evaluation of prototype learning algorithms for nearest-neighbor classifier in application to handwritten character recognition , 2001, Pattern Recognit..

[71]  Andreas Rauber,et al.  Uncovering hierarchical structure in data using the growing hierarchical self-organizing map , 2002, Neurocomputing.

[72]  Jianping Fan,et al.  ClassView: hierarchical video shot classification, indexing, and accessing , 2004, IEEE Transactions on Multimedia.

[73]  Amir F. Atiya,et al.  A penalized likelihood based pattern classification algorithm , 2009, Pattern Recognit..

[74]  Sergios Theodoridis,et al.  Pattern Recognition , 1998, IEEE Trans. Neural Networks.

[75]  Zbigniew W. Ras,et al.  Clustering Driven Cascade Classifiers for Multi-indexing of Polyphonic Music by Instruments , 2010, Advances in Music Information Retrieval.

[76]  Jayanta Kumar Pal,et al.  Spiking problem in monotone regression: Penalized residual sum of squares , 2008 .

[77]  Francisco Javier Ariza-López,et al.  Generalization-oriented road line segmentation by means of an artificial neural network applied over a moving window , 2008, Pattern Recognit..