Minimal Learning Machine: A novel supervised distance-based approach for regression and classification

In this work, a novel supervised learning method, the Minimal Learning Machine (MLM), is proposed. Learning in MLM consists in building a linear mapping between input and output distance matrices. In the generalization phase, the learned distance map is used to provide an estimate of the distance from K output reference points to the unknown target output value. Then, the output estimation is formulated as multilateration problem based on the predicted output distance and the locations of the reference points. Given its general formulation, the Minimal Learning Machine is inherently capable of operating on nonlinear regression problems as well as on multidimensional response spaces. In addition, an intuitive extension of the MLM is proposed to deal with classification problems. A comprehensive set of computer experiments illustrates that the proposed method achieves accuracies that are comparable to more traditional machine learning methods for regression and classification thus offering a computationally valid alternative to such approaches.

[1]  Qinyu. Zhu.  Extreme Learning Machine , 2013 .

[2]  Klaus Obermayer,et al.  A Stochastic Self-Organizing Map for Proximity Data , 1999, Neural Computation.

[3]  D. Marquardt.  An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[4]  Robert J. Schalkoff,et al.  Pattern recognition - statistical, structural and neural approaches , 1991 .

[5]  Jianzhong Wang,et al.  Geometric Structure of High-Dimensional Data and Dimensionality Reduction , 2012 .

[6]  Klaus Obermayer,et al.  Classi cation on Pairwise Proximity , 2007 .

[7]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[8]  Ah Chung Tsoi,et al.  A self-organizing map for adaptive processing of structured data , 2003, IEEE Trans. Neural Networks.

[9]  Alessio Micheli,et al.  A general framework for unsupervised processing of structured data , 2004, Neurocomputing.

[10]  C. M. Cuadras,et al.  A distance based regression model for prediction with mixed data , 1990 .

[11]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[12]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[13]  Brian H. McArdle,et al.  FITTING MULTIVARIATE MODELS TO COMMUNITY DATA: A COMMENT ON DISTANCE‐BASED REDUNDANCY ANALYSIS , 2001 .

[14]  Hui Wang,et al.  Using Radial Basis Function Networks for Function Approximation and Classification , 2012 .

[15]  Vasilios N. Katsikis,et al.  An improved method for the computation of the Moore-Penrose inverse matrix , 2011, Appl. Math. Comput..

[16]  Ewa Niewiadomska-Szynkiewicz,et al.  Optimization Schemes For Wireless Sensor Network Localization , 2009, Int. J. Appl. Math. Comput. Sci..

[17]  Martin D. Buhmann,et al.  Radial Basis Functions , 2021, Encyclopedia of Mathematical Geosciences.

[18]  Panu Somervuo,et al.  Self-Organizing Maps and Learning Vector Quantization for Feature Sequences , 1999, Neural Processing Letters.

[19]  Marc M. Van Hulle,et al.  Enhancing the Yield of High-Density electrode Arrays through Automated electrode Selection , 2012, Int. J. Neural Syst..

[20]  Barbara Hammer,et al.  Topographic Mapping of Large Dissimilarity Data Sets , 2010, Neural Computation.

[21]  Golub Gene H. Et.Al.  Matrix Computations, 3rd Edition , 2007 .

[22]  Frank-Michael Schleif,et al.  Adaptive conformal semi-supervised vector quantization for dissimilarity data , 2014, Pattern Recognit. Lett..

[23]  Jason Weston,et al.  A general regression technique for learning transductions , 2005, ICML '05.

[24]  Horst Bunke.  Structural and Syntactic Pattern Recognition , 1993, Handbook of Pattern Recognition and Computer Vision.

[25]  Joachim M. Buhmann,et al.  Pairwise Data Clustering by Deterministic Annealing , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  F. Wilcoxon.  Individual Comparisons by Ranking Methods , 1945 .

[27]  Gene H. Golub,et al.  Matrix computations , 1983 .

[28]  Frank-Michael Schleif,et al.  Learning vector quantization for (dis-)similarities , 2014, Neurocomputing.

[29]  Thomas Hofmann,et al.  Pattern Recognition, Statistical , 2006 .

[30]  Amaury Lendasse,et al.  OP-ELM: Optimally Pruned Extreme Learning Machine , 2010, IEEE Transactions on Neural Networks.

[31]  Frank-Michael Schleif,et al.  Linear Time Relational Prototype Based Learning , 2012, Int. J. Neural Syst..

[32]  Pierre Courrieu,et al.  Fast Computation of Moore-Penrose Inverse Matrices , 2008, ArXiv.

[33]  G. Warnock,et al.  Thinking About Thinking , 1975 .

[34]  Jeremy W. Lichstein,et al.  Multiple regression on distance matrices: a multivariate spatial analysis tool , 2007, Plant Ecology.

[35]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[36]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[37]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[38]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[39]  Bernhard Schölkopf,et al.  Kernel Dependency Estimation , 2002, NIPS.

[40]  Ponnuthurai Nagaratnam Suganthan,et al.  A novel kernel prototype-based learning algorithm , 2004, ICPR 2004.

[41]  Bernard Carlos Widrow,et al.  Thinking about thinking: the discovery of the LMS algorithm , 2005, IEEE Signal Process. Mag..

[42]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[43]  Willy Hereman,et al.  Statistical methods in surveying by trilateration , 1998 .

[44]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[45]  Thomas Villmann,et al.  Efficient Kernelized Prototype Based Classification , 2011, Int. J. Neural Syst..