A unified SWSI-KAMs framework and performance evaluation on face recognition

The Kernel method is an effective and popular trick in machine learning. In this paper, by introducing it into conventional auto-associative memory models (AMs), we construct a unified framework of kernel auto-associative memory models (KAMs), which makes the existing exponential and polynomial AMs become its special cases. Further, in order to reduce KAM's connect complexity, inspired by ''small-world network'' recently described by Watts and Strogatz, we propose another unified framework of small-world structure (SWS) inspired kernel auto-associative memory models (SWSI-KAMs), which, in principle, makes KAMs simpler in structure. Simulation results on the FERET face database show that, the SWSI-KAMs adopting kernels such as Exponential and Hyperbolic tangent kernels have advantages of configuration simplicity while their recognition performance is almost as good as or even better than corresponding KAMs with full connectivity. In the end, the SWSI-KAM adopting Exponential kernel with different connectivities was emphatically investigated for robustness based on those face images which were added random noises and/or partially occluded in a mosaic way, and the experiments demonstrate that the SWSI-KAM with Exponential kernel is more robust in all cases of network connectivity of 20%, 40% and 60% than both PCA and recently proposed (PC)^2A algorithms for face recognition. cognition.

[1]  Tzi-Dar Chiueh,et al.  Multivalued associative memories based on recurrent networks , 1993, IEEE Trans. Neural Networks.

[2]  Harry Wechsler,et al.  The FERET database and evaluation procedure for face-recognition algorithms , 1998, Image Vis. Comput..

[3]  Zeng-ou Wang.  A Bidirectional Associative Memory Based on Optimal Linear Associative Memory , 1996, IEEE Trans. Computers.

[4]  A. Rbnyi.  ON THE EVOLUTION OF RANDOM GRAPHS , 2001 .

[5]  Jianxin Wu,et al.  Face recognition with one training image per person , 2002, Pattern Recognit. Lett..

[6]  Adam Blum,et al.  Bidirectional associative memory systems in C , 1990 .

[7]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[8]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[9]  Chung-Ping Kwong,et al.  Recurrent correlation associative memories with multiple-value , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[10]  BART KOSKO,et al.  Bidirectional associative memories , 1988, IEEE Trans. Syst. Man Cybern..

[11]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[12]  János Komlós,et al.  Effect of Connectivity in an Associative Memory Model , 1993, J. Comput. Syst. Sci..

[13]  Andrew Canning,et al.  Partially connected models of neural networks , 1988 .

[14]  Rodney M. Goodman,et al.  Recurrent correlation associative memories , 1991, IEEE Trans. Neural Networks.

[15]  Tong Zhang,et al.  An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods , 2001, AI Mag..

[16]  Beom Jun Kim.  Performance of networks of artificial neurons: the role of clustering. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[17]  Sharon L. Milgram,et al.  The Small World Problem , 1967 .

[18]  Tzi-Dar Chiueh,et al.  Exponential bidirectional associative memories , 1990 .

[19]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[20]  Ali A. Minai,et al.  Efficient associative memory using small-world architecture , 2001, Neurocomputing.

[21]  Santosh S. Venkatesh,et al.  The capacity of the Hopfield associative memory , 1987, IEEE Trans. Inf. Theory.

[22]  Santosh S. Venkatesh.  Robustness in neural computation: Random graphs and sparsity , 1992, IEEE Trans. Inf. Theory.

[23]  Masafumi Hagiwara,et al.  Intersection Learning for Bidirectional Associative Memory , 1996 .

[24]  B. Bollobás.  The evolution of random graphs , 1984 .

[25]  Daoqiang Zhang,et al.  Robust image segmentation using FCM with spatial constraints based on new kernel-induced distance measure , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[26]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[27]  Heng-Ming Tai,et al.  High-order bidirectional associative memory , 1989 .

[28]  Michael Menzinger,et al.  Topology and computational performance of attractor neural networks. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[29]  Joaquín J. Torres,et al.  Influence of topology on the performance of a neural network , 2004, Neurocomputing.

[30]  János Komlós,et al.  Effect of connectivity in associative memory models , 1988, [Proceedings 1988] 29th Annual Symposium on Foundations of Computer Science.

[31]  Dao-Qiang Zhang,et al.  Clustering Incomplete Data Using Kernel-Based Fuzzy C-means Algorithm , 2003, Neural Processing Letters.

[32]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[33]  Heekuck Oh,et al.  Adaptation of the relaxation method for learning in bidirectional associative memory , 1994, IEEE Trans. Neural Networks.

[34]  Konstantinos N. Plataniotis,et al.  Face recognition using kernel direct discriminant analysis algorithms , 2003, IEEE Trans. Neural Networks.

[35]  P. Erdos,et al.  On the evolution of random graphs , 1984 .

[36]  E. Capaldi,et al.  The organization of behavior. , 1992, Journal of applied behavior analysis.

[37]  William S. Yamamoto,et al.  AY's Neuroanatomy of C. elegans for Computation , 1992 .

[38]  Bo Zhang,et al.  Programming based learning algorithms of neural networks with self-feedback connections , 1995, IEEE Trans. Neural Networks.

[39]  Harris Drucker,et al.  Comparison of learning algorithms for handwritten digit recognition , 1995 .

[40]  Wei Yan,et al.  Improved exponential bidirectional associative memory , 1997 .

[41]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[42]  Wen-June Wang,et al.  A multivalued bidirectional associative memory operating on a complex domain , 1998, Neural Networks.

[43]  Alex Pentland,et al.  Face recognition using eigenfaces , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.