Neural Networks from Similarity Based Perspective

Abstract: A framework for Similarity-Based Methods (SBMs) includes many neural network models as special cases. Multilayer Perceptrons (MLPs) use scalar products to compute weighted activation of neurons, combining soft hyperplanes to provide decision borders. Scalar product is replaced by a distance function between the inputs and the weights, offering a natural generalization of the standard MLP model to the distance-based multilayer perceptron (D-MLP) model. D-MLPs evaluate similarity of inputs to weights making the interpretation of their mappings easier. Cluster-based initialization procedure determining architecture and values of all adaptive parameters is described. D-MLP networks are useful not only for classification and approximation, but also as associative memories, in problems requiring pattern completion, offering an efficient way to deal with missing values. Non-Euclidean distance functions may also be introduced by normalization of the input vectors in an extended feature space. Both approaches influence the shapes of decision borders dramatically. An illustrative example showing these changes is provided.

[1]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[2]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[3]  R. Lippmann,et al.  An introduction to computing with neural nets , 1987, IEEE ASSP Magazine.

[4]  Sandro Ridella,et al.  Circular backpropagation networks for classification , 1997, IEEE Trans. Neural Networks.

[5]  G. Dorffner.  UNIFIED FRAMEWORK FOR MLPs AND RBFNs: INTRODUCING CONIC SECTION FUNCTION NETWORKS , 1994 .

[6]  Jean-Pierre Martens,et al.  On the initialization and optimization of multilayer perceptrons , 1994, IEEE Trans. Neural Networks.

[7]  Wlodzislaw Duch,et al.  Feature space mapping as a universal adaptive system , 1995 .

[8]  David M. Skapura,et al.  Building neural networks , 1995 .

[9]  Włodzisław Duch,et al.  Neural minimal distance methods , 1997 .

[10]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[11]  Krzysztof Grabczewski,et al.  Extraction of logical rules from backpropagation networks , 1998 .

[12]  L. N. Kanal,et al.  Handbook of Statistics, Vol. 2. Classification, Pattern Recognition and Reduction of Dimensionality. , 1985 .

[13]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[14]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[15]  R. Miranda,et al.  Circular Nodes in Neural Networks , 1996, Neural Computation.

[16]  Wlodzislaw Duch,et al.  A new methodology of extraction, optimization and application of crisp and fuzzy logical rules , 2001, IEEE Trans. Neural Networks.

[17]  D. Rubin.  Multiple Imputation After 18+ Years , 1996 .