Guaranteed storage and stabilization of desired binary periodic orbits in three-layer dynamic binary neural networks

Abstract This paper presents three-layer dynamic binary neural networks characterized by ternary connection parameters and the signum activation function. The dynamics is described by a difference equation of binary state variables. Depending on the parameters, the network can generate various binary periodic orbits. We give two main theoretical results. First, when a desired periodic orbit is given, we can set the parameters that guarantee storage and local stability of the periodic orbit. The stability is related to error correction of various binary signals in engineering applications. Second, if a part of the connection parameters becomes zero then stability of the periodic orbit becomes very strong. In this case, all the initial states fall directly into the periodic orbit.

[1]  Toshimichi Saito,et al.  An associative memory including time-variant self-feedback , 1994, Neural Networks.

[2]  Daizhan Cheng,et al.  Control of Large-Scale Boolean Networks via Network Aggregation , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Marco Tomassini,et al.  On the Generation of High-Quality Random Numbers by Two-Dimensional Cellular Automata , 2000, IEEE Trans. Computers.

[4]  Yoshua Bengio,et al.  BinaryConnect: Training Deep Neural Networks with binary weights during propagations , 2015, NIPS.

[5]  Zhigang Zeng,et al.  Multiperiodicity of Discrete-Time Delayed Neural Networks Evoked by Periodic External Inputs , 2006, IEEE Transactions on Neural Networks.

[6]  Anthony N. Michel,et al.  A training algorithm for binary feedforward neural networks , 1992, IEEE Trans. Neural Networks.

[7]  Frank Pasemann,et al.  Characterization of periodic attractors in neural ring networks , 1995, Neural Networks.

[8]  Leon O. Chua.  A Nonlinear Dynamics Perspective Of Wolfram'S New Kind Of Science , 2013 .

[9]  Toshimichi Saito,et al.  Theoretical analysis of dynamic binary neural networks with simple sparse connection , 2019, Neurocomputing.

[10]  Zhiming Zheng,et al.  The stability of Boolean network with transmission sensitivity , 2017 .

[11]  Zhou Zhaoying,et al.  Approximation to Boolean functions by neural networks with applications to thinning algorithms , 2000, Proceedings of the 17th IEEE Instrumentation and Measurement Technology Conference [Cat. No. 00CH37066].

[12]  Shigetoshi Nara,et al.  Completely reproducible description of digital sound data with cellular automata , 2002 .

[13]  Vincent Gripon,et al.  Storing Sequences in Binary Tournament-Based Neural Networks , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Dario Floreano,et al.  Climbing favours the tripod gait over alternative faster insect gaits , 2017, Nature Communications.

[15]  Wei Wu,et al.  Binary Higher Order Neural Networks for Realizing Boolean Functions , 2011, IEEE Transactions on Neural Networks.

[16]  Toshimichi Saito,et al.  Stabilization of desired periodic orbits in dynamic binary neural networks , 2017, Neurocomputing.

[17]  Parimal Pal Chaudhuri,et al.  Design of CAECC-Cellular Automata Based Error Correcting Code , 1994, IEEE Trans. Computers.

[18]  Toshimichi Saito,et al.  Learning of Simple Dynamic Binary Neural Networks , 2013, IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences.

[19]  Katsutoshi Saeki,et al.  Central Pattern Generator Based on Interstitial Cell Models Made from Bursting Neuron Models , 2018, ICONIP.

[20]  Ruedi Stoop,et al.  A full computation-relevant topological dynamics classification of elementary cellular automata. , 2012, Chaos.