Analysis and learning of periodic orbits in dynamic binary neural networks

This paper studies the dynamic binary neural network (DBNN) that can generate a variety of binary periodic orbits. The DBNN is constructed by applying the delayed feedback to a three-layer network. It is characterized by the signum activation function and ternary/binary weighting parameters. First, we give a systematic analysis tool: the Gray-code-based return map that is useful to grasp basic characteristics of the DBNN such as the number of periodic orbits and their domain of attraction. Second, we show that the DBNN includes an equivalent system of the cellular automata: this fact encourages study of the DBNN. Third, applying a learning algorithm to a teacher signal of periodic orbit, we have confirmed storage of the teacher signal, generation of a different periodic orbit and automatic stabilization of the periodic orbits.

[1]  Toshimichi Saito,et al.  Digital spiking neuron and its learning for approximation of various spike-trains , 2008, Neural Networks.

[2]  Shigetoshi Nara,et al.  Completely reproducible description of digital sound data with cellular automata , 2002 .

[3]  Anthony N. Michel,et al.  A training algorithm for binary feedforward neural networks , 1992, IEEE Trans. Neural Networks.

[4]  Bimal K. Bose,et al.  Neural Network Applications in Power Electronics and Motor Drives—An Introduction and Perspective , 2007, IEEE Transactions on Industrial Electronics.

[5]  Wei Wu,et al.  Binary Higher Order Neural Networks for Realizing Boolean Functions , 2011, IEEE Transactions on Neural Networks.

[6]  Marcin Seredynski,et al.  Block cipher based on reversible cellular automata , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[7]  Toshimichi Saito,et al.  Dynamic binary neural networks and evolutionary learning , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[8]  Guanrong Chen,et al.  Universal Perceptron and DNA-Like Learning Algorithm for Binary Neural Networks: Non-LSBF Implementation , 2009, IEEE Transactions on Neural Networks.

[9]  Paul L. Rosin.  Training cellular automata for image processing , 2005, IEEE Transactions on Image Processing.

[10]  Toshimichi Saito,et al.  Switched dynamical systems with double periodic inputs: an analysis tool and its application to the buck-boost converter , 2000 .

[11]  M. Shimada,et al.  A simple learning of binary neural networks with virtual teacher signals , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[12]  D. Liberati,et al.  Training digital circuits with Hamming clustering , 2000 .

[13]  Guolong He,et al.  Universal Perceptron and DNA-Like Learning Algorithm for Binary Neural Networks: LSBF and PBF Implementations , 2009, IEEE Transactions on Neural Networks.

[14]  E. Ott.  Chaos in Dynamical Systems: Contents , 1993 .

[15]  James A. Reggia,et al.  Automatic discovery of self-replicating structures in cellular automata , 1997, IEEE Trans. Evol. Comput..

[16]  Sung-Kwon Park,et al.  The geometrical learning of binary neural networks , 1995, IEEE Trans. Neural Networks.

[17]  T. Saitô,et al.  A Flexible Learning Algorithm for Binary Neural Networks , 1998 .

[18]  Leon O. Chua.  A Nonlinear Dynamics Perspective Of Wolfram'S New Kind Of Science , 2013 .

[19]  Toshimichi Saito,et al.  Learning of Dynamic BNN toward Storing-and-Stabilizing Periodic Patterns , 2011, ICONIP.

[20]  Toshimichi Saito,et al.  A Simple Class of Binary Neural Networks and Logical Synthesis , 2011, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..