PRSOM: a new visualization method by hybridizing multidimensional scaling and self-organizing map

Self-organizing map (SOM) is an approach of nonlinear dimension reduction and can be used for visualization. It only preserves topological structures of input data on the projected output space. The interneuron distances of SOM are not preserved from input space into output space such that the visualization of SOM can be degraded. Visualization-induced SOM (ViSOM) has been proposed to overcome this problem. However, ViSOM is derived from heuristic and no cost function is assigned to it. In this paper, a probabilistic regularized SOM (PRSOM) is proposed to give a better visualization effect. It is associated with a cost function and gives a principled rule for weight-updating. The advantages of both multidimensional scaling (MDS) and SOM are incorporated in PRSOM. Like MDS, The interneuron distances of PRSOM in input space resemble those in output space, which are predefined before training. Instead of the hard assignment by ViSOM, the soft assignment by PRSOM can be further utilized to enhance the visualization effect. Experimental results demonstrate the effectiveness of the proposed PRSOM method compared with other dimension reduction methods.

[1]  Tommy W. S. Chow,et al.  Induction machine fault detection using SOM-based RBF neural networks , 2004, IEEE Transactions on Industrial Electronics.

[2]  M. Kramer.  Nonlinear principal component analysis using autoassociative neural networks , 1991 .

[3]  Charles E. Heckler,et al.  Applied Multivariate Statistical Analysis , 2005, Technometrics.

[4]  Helge J. Ritter,et al.  Neural computation and self-organizing maps - an introduction , 1992, Computation and neural systems series.

[5]  Terence D. Sanger,et al.  Optimal unsupervised learning in a single-layer linear feedforward neural network , 1989, Neural Networks.

[6]  Klaus Obermayer,et al.  Self-organizing maps: Generalizations and new optimization techniques , 1998, Neurocomputing.

[7]  Vladimir Cherkassky,et al.  Self-Organization as an Iterative Kernel Smoothing Process , 1995, Neural Computation.

[8]  J. Kruskal.  Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis , 1964 .

[9]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[10]  Aluizio F. R. Araújo,et al.  Identification and control of dynamical systems using the self-organizing map , 2004, IEEE Transactions on Neural Networks.

[11]  J. Rubner,et al.  A Self-Organizing Network for Principal-Component Analysis , 1989 .

[12]  Juha Karhunen,et al.  Generalizations of principal component analysis, optimization problems, and neural networks , 1995, Neural Networks.

[13]  Anil K. Jain,et al.  Artificial neural networks for feature extraction and multivariate data projection , 1995, IEEE Trans. Neural Networks.

[14]  Erkki Oja,et al.  S-Map: A Network with a Simple Self-Organization Algorithm for Generative Topographic Mappings , 1997, NIPS.

[15]  Tommy W. S. Chow,et al.  Content-based image retrieval using growing hierarchical self-organizing quadtree map , 2005, Pattern Recognit..

[16]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[17]  K. Obermayer,et al.  PHASE TRANSITIONS IN STOCHASTIC SELF-ORGANIZING MAPS , 1997 .

[18]  Hujun Yin,et al.  Data visualisation and manifold mapping using the ViSOM , 2002, Neural Networks.

[19]  M. V. Velzen,et al.  Self-organizing maps , 2007 .

[20]  R. Tibshirani,et al.  Adaptive Principal Surfaces , 1994 .

[21]  Nikhil R. Pal,et al.  Two efficient connectionist schemes for structure preserving dimensionality reduction , 1998, IEEE Trans. Neural Networks.

[22]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[23]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[24]  Mu-Chun Su,et al.  A new model of self-organizing neural networks and its application in data projection , 2001, IEEE Trans. Neural Networks.

[25]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[26]  Yizong Cheng.  Convergence and Ordering of Kohonen's Batch Map , 1997, Neural Computation.

[27]  O. Mangasarian,et al.  Multisurface method of pattern separation for medical diagnosis applied to breast cytology. , 1990, Proceedings of the National Academy of Sciences of the United States of America.

[28]  Anil K. Jain,et al.  A nonlinear projection method based on Kohonen's topology preserving maps , 1992, IEEE Trans. Neural Networks.

[29]  Chip-Hong Chang,et al.  New adaptive color quantization method based on self-organizing maps , 2005, IEEE Transactions on Neural Networks.

[30]  Yanda Li,et al.  Self-organizing map as a new method for clustering and data analysis , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[31]  Donald B. Rubin,et al.  Max-imum Likelihood from Incomplete Data , 1972 .

[32]  Jeanny Hérault,et al.  Curvilinear component analysis: a self-organizing neural network for nonlinear mapping of data sets , 1997, IEEE Trans. Neural Networks.

[33]  Hujun Yin,et al.  ViSOM - a novel method for multivariate data projection and structure visualization , 2002, IEEE Trans. Neural Networks.

[34]  A. Konig.  Interactive visualization and analysis of hierarchical neural projections for data mining , 2000 .

[35]  Erkki Oja,et al.  Neural Networks, Principal Components, and Subspaces , 1989, Int. J. Neural Syst..

[36]  T. Hastie,et al.  Principal Curves , 2007 .

[37]  John W. Sammon,et al.  A Nonlinear Mapping for Data Structure Analysis , 1969, IEEE Transactions on Computers.

[38]  Andreas König,et al.  Interactive visualization and analysis of hierarchical neural projections for data mining , 2000, IEEE Trans. Neural Networks Learn. Syst..

[39]  Akira Hirose,et al.  Predictive self-organizing map for vector quantization of migratory signals and its application to mobile communications , 2003, IEEE Trans. Neural Networks.

[40]  Shen Jun,et al.  A Hierarchical Self-Organizing Approach for Learning the Patterns of Motion Trajectories , 2003 .