Transformations of sigma-pi nets: obtaining reflected functions by reflecting weight matrices

This paper presents a methodology that reflected functions by reflecting the weight matrices of an artificial neural network. One of the major problems with the connectionist approach is that trained neural networks can only associate fixed sets of input-output mappings. We provide a methodology which allows the post-trained net to associate different input-output mappings. The different mappings are reflected in a horizontal axis, reflected in a vertical axis and scaling of the initial mapping. The methodology does not train the net on the different mappings but it transforms the weight matrix of the neural network. This paper describes a novel way of utilising sigma-pi neural networks. Our new methodology manipulates sigma-pi unit's weight matrices which transform the unit's output. The weights are cast in a matrix formulation, and then transformations can be performed on the weight matrix of the sigma-pi net. To test the new methodology, the following three steps were carried out on a neural network: (1) the network was trained to perform a mapping function, f; (2) the weights of the network were transformed; and (3) the network was tested to evaluate whether it performs the reflection in the vertical axis,f(ref-vert)(x) = a - f(x). This reflects the function in one dimension. A reflection transformation was used to manipulate the network's weight matrices to obtain a reflection in the vertical axis. Note that the network was not trained to perform the reflection in the vertical axis. The transformation of the weight matrix transformed the function the output performs. This article explains the theory which enables us to perform transformations of sigma-pi networks and obtain reflections of the output by reflecting the weight matrices. These transforms empower the network to perform related mapping tasks once one mapping task has been learnt. This article explains how each transformation is performed and it considers whether a set of 'standard' transformations can indeed be derived.

[1]  T. J. Stonham,et al.  Generalization in Sigma-Pi Networks , 1995 .

[2]  Richard Stuart Neville.  Partially Precalculated Weights for Backpropagation Training of RAM-Based sigma-pi Nets , 1997, SCAI.

[3]  R. Unbehauen,et al.  Neural networks for solving systems of linear equations. II. Minimax and least absolute value problems , 1992 .

[4]  Jun Wang,et al.  A recurrent neural network for solving linear projection equations , 2000, Neural Networks.

[5]  Nikola Samardzija,et al.  A neural network for computing eigenvectors and eigenvalues , 1991, Biological Cybernetics.

[6]  Hua Li,et al.  Solving Simultaneous Linear Equations Using Recurrent Neural Networks , 1994, Inf. Sci..

[7]  Jerry M. Mendel,et al.  Structured trainable networks for matrix algebra , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[8]  R. S. Neville.  RAM-based Sigma-pi nets for high accuracy function mapping , 1997 .

[9]  Alison Cawsey,et al.  The essence of artificial intelligence , 1997 .

[10]  Jun Wang,et al.  Recurrent neural networks for LU decomposition and Cholesky factorization , 1993 .

[11]  K. A. Stroud.  Engineering Mathematics: Programmes and Problems , 1970 .

[12]  Jun Wang,et al.  A multilayer recurrent neural network for solving continuous-time algebraic Riccati equations , 1998, Neural Networks.

[13]  Marimuthu Palaniswami,et al.  Computational Intelligence: A Dynamic System Perspective , 1995 .

[14]  K. A. Stroud.  Further Engineering Mathematics: Programmes and Problems , 1990 .

[15]  Dimitrios I. Fotiadis,et al.  Artificial neural networks for solving ordinary and partial differential equations , 1997, IEEE Trans. Neural Networks.

[16]  Kevin N. Gurney.  Learning in networks of structured hypercubes , 1989 .

[17]  Kevin N. Gurney,et al.  Training nets of hardware realizable sigma-pi units , 1992, Neural Networks.

[18]  Konrad Reif,et al.  Multilayer neural networks for solving a class of partial differential equations , 2000, Neural Networks.

[19]  Jerry M. Mendel,et al.  Parallel Structured Networks for Solving a Wide Variety of Matrix Algebra Problems , 1992, J. Parallel Distributed Comput..

[20]  Andrew J. Meade,et al.  The numerical solution of linear ordinary differential equations by feedforward neural networks , 1994 .

[21]  Jun Wang,et al.  A recurrent neural network for real-time matrix inversion , 1993 .

[22]  Luo Fa-long,et al.  Neural network approach to computing matrix inversion , 1992 .

[23]  T. John Stonham,et al.  Adaptive associative reward-penalty algorithms for sigma-pi networks , 1994, Neural Parallel Sci. Comput..

[24]  T. John Stonham,et al.  Partially pre-calculated weights for the backpropagation learning regime and high accuracy function mapping using continuous input RAM-based sigma-pi nets , 2000, Neural Networks.

[25]  T. John Stonham,et al.  Adaptive critic for sigma-pi networks , 1996, Neural Networks.

[26]  Jun Wang,et al.  Electronic realisation of recurrent neural network for solving simultaneous linear equations , 1992 .

[27]  Hamid Bolouri,et al.  Learning algorithms for RAM-based neural networks , 1997 .

[28]  P.C.K. Luk,et al.  Toward second-order generalisation , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[29]  Jim Austin.  A review of RAM based neural networks , 1994, Proceedings of the Fourth International Conference on Microelectronics for Neural Networks and Fuzzy Systems.

[30]  D. Gorse,et al.  Training Strategies for Probabilistic RAMs , 1992 .

[31]  Jerry M. Mendel,et al.  Three-Dimensional Structured Networks for Matrix Equation Solving , 1991, IEEE Trans. Computers.

[32]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[34]  John G. Taylor,et al.  Learning Probabilistic RAM Nets Using VLSI Structures , 1992, IEEE Trans. Computers.

[35]  Kevin Gurney.  Weighted Nodes and RAM-Nets: A Unified Approach , 1992 .

[36]  A. Cichocki,et al.  Neural networks for solving systems of linear equations and related problems , 1992 .

[37]  Soo-Young Lee,et al.  An Optimization Network for Matrix Inversion , 1987, NIPS.