Model of an oscillatory neural network with multilevel neurons for pattern recognition

The current study uses a novel method of multilevel neurons and high order synchronization effects described by a family of special metrics, for pattern recognition in an oscillatory neural network (ONN). The output oscillator (neuron) of the network has multilevel variations in its synchronization value with the reference oscillator, and allows classification of an input pattern into a set of classes. The ONN model is implemented on thermally-coupled vanadium dioxide oscillators. The ONN is trained by the simulated annealing algorithm for selection of the network parameters. The results demonstrate that ONN is capable of classifying 512 visual patterns (as a cell array 3 × 3, distributed by symmetry into 102 classes) into a set of classes with a maximum number of elements up to fourteen. The classification capability of the network depends on the interior noise level and synchronization effectiveness parameter. The model allows for designing multilevel output cascades of neural networks with high net data throughput. The presented method can be applied in ONNs with various coupling mechanisms and oscillator topology.

[1]  Vladimir I. Nekorkin,et al.  Synchronization of delay-coupled oscillator networks , 2013 .

[2]  R. Roy,et al.  Experimental observation of chimeras in coupled-map lattices , 2012, Nature Physics.

[3]  Christoph von der Malsburg,et al.  The Correlation Theory of Brain Function , 1994 .

[4]  Vijaykrishnan Narayanan,et al.  Synchronized charge oscillations in correlated electron systems , 2014, Scientific Reports.

[5]  Carson C. Chow,et al.  Synchronization and Oscillatory Dynamics in Heterogeneous, Mutually Inhibited Neurons , 1998, Journal of Computational Neuroscience.

[6]  Alexander Pergament,et al.  Electron beam modification of vanadium dioxide oscillators , 2016 .

[7]  Tadashi Shibata,et al.  Coupled-Oscillator Associative Memory Array Operation for Pattern Recognition , 2015, IEEE Journal on Exploratory Solid-State Computational Devices and Circuits.

[8]  Dimitri M. Kullmann,et al.  Oscillations and Filtering Networks Support Flexible Routing of Information , 2010, Neuron.

[9]  Robert Callan,et al.  The essence of neural networks , 1998 .

[10]  Mahmut Ozer,et al.  Dynamical structure underlying inverse stochastic resonance and its implications. , 2013, Physical review. E, Statistical, nonlinear, and soft matter physics.

[11]  J. Sakai,et al.  High-efficiency voltage oscillation in VO2 planer-type junctions with infinite negative differential resistance , 2008 .

[12]  Suman Datta,et al.  Modeling and Simulation of Vanadium Dioxide Relaxation Oscillators , 2015, IEEE Transactions on Circuits and Systems I: Regular Papers.

[13]  V. Cros,et al.  Spin-torque building blocks. , 2014, Nature Materials.

[14]  Dominique M. Durand,et al.  Stochastic Resonance Can Enhance Information Transmission in Neural Networks , 2011, IEEE Transactions on Biomedical Engineering.

[15]  W. Freeman.  Spatial properties of an EEG event in the olfactory bulb and cortex. , 1978, Electroencephalography and clinical neurophysiology.

[16]  Jürgen Kurths,et al.  Synchronization - A Universal Concept in Nonlinear Sciences , 2001, Cambridge Nonlinear Science Series.

[17]  Damien Querlioz,et al.  Vowel recognition with four coupled spin-torque nano-oscillators , 2017, Nature.

[18]  Carson C. Chow,et al.  Stochastic resonance without tuning , 1995, Nature.

[19]  Damien Querlioz,et al.  A Nanotechnology-Ready Computing Scheme based on a Weakly Coupled Oscillator Network , 2017, Scientific Reports.

[20]  T. C. L. G. Sollner,et al.  Resonant-tunneling-diode relaxation oscillator☆ , 2000 .

[21]  Alexander Pergament,et al.  The effect of electric field on metal-insulator phase transition in vanadium dioxide , 2002 .

[22]  James A. Bain,et al.  Phase Coupling and Control of Oxide-Based Oscillators for Neuromorphic Computing , 2015, IEEE Journal on Exploratory Solid-State Computational Devices and Circuits.

[23]  Alexander Pergament,et al.  Deterministic noise in vanadium dioxide based structures , 2003 .

[24]  Suman Datta,et al.  Exploiting Synchronization Properties of Correlated Electron Devices in a Non-Boolean Computing Fabric for Template Matching , 2014, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[25]  E. Parzen,et al.  Modern Probability Theory and Its Applications , 1960 .

[26]  S. D. Khanin,et al.  Switching Channel Development Dynamics in Planar Structures on the Basis of Vanadium Dioxide , 2018 .

[27]  Alexander Pergament,et al.  Thermal coupling and effect of subharmonic synchronization in a system of two VO2 based oscillators , 2020, ArXiv.

[28]  Fernando Corinto,et al.  Weakly Connected Oscillatory Network Models for Associative and Dynamic Memories , 2007, Int. J. Bifurc. Chaos.

[29]  Irina Surina,et al.  Oscillatory network with self-organized dynamical connections for synchronization-based image segmentation. , 2004, Bio Systems.

[30]  Frank C. Hoppensteadt,et al.  Pattern recognition via synchronization in phase-locked loop neural networks , 2000, IEEE Trans. Neural Networks Learn. Syst..

[31]  Julie Wall,et al.  Solving the linearly inseparable XOR problem with spiking neural networks , 2017, 2017 Computing Conference.

[32]  Alexander Pergament,et al.  Switching dynamics of single and coupled VO2-based oscillators as elements of neural networks , 2017, ArXiv.

[33]  Patrick Suppes,et al.  Learning Pattern Recognition Through Quasi-Synchronization of Phase Oscillators , 2011, IEEE Transactions on Neural Networks.

[34]  Juan P. Torres,et al.  The Kuramoto model: A simple paradigm for synchronization phenomena , 2005 .

[35]  G Bard Ermentrout,et al.  Intrinsic heterogeneity in oscillatory dynamics limits correlation-induced neural synchronization. , 2012, Journal of neurophysiology.

[36]  S. Datta,et al.  Pairwise coupled hybrid vanadium dioxide-MOSFET (HVFET) oscillators for non-boolean associative computing , 2014, 2014 IEEE International Electron Devices Meeting.

[37]  R. Eckhorn,et al.  Coherent oscillations: A mechanism of feature linking in the visual cortex? , 1988, Biological Cybernetics.

[38]  S. Ghosh.  Generation of high-frequency power oscillation by astable mode arcing with SCR switched inductor , 1984 .

[39]  Maksim Belyaev,et al.  A New Method of the Pattern Storage and Recognition in Oscillatory Neural Networks Based on Resistive Switches , 2018, Electronics.

[40]  W. Singer,et al.  Stimulus-specific neuronal oscillations in orientation columns of cat visual cortex. , 1989, Proceedings of the National Academy of Sciences of the United States of America.

[41]  Alexander Pergament,et al.  Modeling of thermal coupling in VO2-based oscillatory neural networks , 2018 .

[42]  Suman Datta,et al.  Neuro inspired computing with coupled relaxation oscillators , 2014, 2014 51st ACM/EDAC/IEEE Design Automation Conference (DAC).

[43]  Andrey Velichko,et al.  Effects of Higher Order and Long-Range Synchronizations for Classification and Computing in Oscillator-Based Spiking Neural Networks , 2018, ArXiv.