Implementing delay insensitive oscillatory neural networks using CMOS and emerging technology

One major challenge in efficiently implementing neuromorphic networks is the need for a large number of variable synaptic connections. Networks that use emerging resistive memories as synapses have been proposed to tackle this problem, but interfacing with these devices is still inefficient in deeply-scaled CMOS. Oscillatory Neural Networks (ONNs) use a different paradigm than most analog hardware implementations, and may be able to interface more efficiently with RRAM neurons. Previous work on ONNs, however, has not considered the effects of actual hardware implementation realities, such as delay in the network. In this work, the first reported IC implementation of an oscillatory neural network is designed and fabricated. Modifications are made to the ONN architecture based on theoretical analysis to allow for proper operation in real-world conditions. One modification is changing the PLL-type, giving the system a different dynamic trajectory which is robust to global delays. Additionally, circuitry is added to control the transport delay of the neuron output signals. A chip with the modified ONN architecture is designed and tested in 28 nm CMOS and estimated power and area figures are reported.

[1]  M. Prezioso,et al.  RRAM-based hardware implementations of artificial neural networks: progress update and challenges ahead , 2016, SPIE OPTO.

[2]  James A. Bain,et al.  Oscillatory Neural Networks Based on TMO Nano-Oscillators and Multi-Level RRAM Cells , 2015, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[3]  Navdeep Jaitly,et al.  Towards End-To-End Speech Recognition with Recurrent Neural Networks , 2014, ICML.

[4]  Soummya Kar,et al.  On the design of phase locked loop oscillatory neural networks: Mitigation of transmission delay effects , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[5]  Akira Goda Recent progress and future directions in NAND Flash scaling , 2013, 2013 13th Non-Volatile Memory Technology Symposium (NVMTS).

[6]  Ángel Rodríguez-Vázquez,et al.  ACE4k: An analog I/O 64×64 visual microprocessor chip with 7-bit analog accuracy , 2002, Int. J. Circuit Theory Appl..

[7]  G. W. Burr,et al.  Experimental demonstration and tolerancing of a large-scale neural network (165,000 synapses), using phase-change memory as the synaptic weight element , 2015, 2014 IEEE International Electron Devices Meeting.

[8]  Ángel Rodríguez-Vázquez,et al.  ACE4k: An analog I/O 64×64 visual microprocessor chip with 7-bit analog accuracy: Research Articles , 2002 .

[9]  Steven R. Young,et al.  A 1 TOPS/W Analog Deep Machine-Learning Engine With Floating-Gate Storage in 0.13 µm CMOS , 2014, IEEE Journal of Solid-State Circuits.

[10]  Frank C. Hoppensteadt,et al.  Pattern recognition via synchronization in phase-locked loop neural networks , 2000, IEEE Trans. Neural Networks Learn. Syst..

[11]  Fei-Fei Li,et al.  Large-Scale Video Classification with Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Behzad Razavi,et al.  The StrongARM Latch [A Circuit for All Seasons] , 2015, IEEE Solid-State Circuits Magazine.

[13]  Johannes Schemmel,et al.  Wafer-scale integration of analog neural networks , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).