A System for Validating Resistive Neural Network Prototypes

Building prototypes of heterogeneous hardware systems based on emerging electronic, magnetic, and photonic devices is an important area of research. The novel implementation of these systems for artificial intelligence poses new and unforeseen challenges in mixed signal data acquisition, hyperparameter optimization, and hardware co-processing. Many emerging devices exhibit unpredictable and stochastic behavior as well as poorly repeatable hysteretic effects or performance degradation. Dealing with these device challenges on top of more traditional hardware problems, like quantization errors, timing constraints, and even hardware and software bugs is an enterprise fraught with pitfalls. Equally important to the construction of the physical prototype is the co-development and integration of a design verification framework that can extensibly allow for predictable behavior of not only the entire system but also all of its parts in a modular way, allowing for seamless integration in both simulation and implementation. This work discusses Daffodil-lib, a Python based prototyping framework which, from hardware to software, enables everything from a script-based simulation to a compiled hardware-timed experiment, to everything in between with no syntactical changes for the end user.

[1]  David Blaauw,et al.  Analog in-memory subthreshold deep neural network accelerator , 2017, 2017 IEEE Custom Integrated Circuits Conference (CICC).

[2]  Xiaochen Peng,et al.  NeuroSim: A Circuit-Level Macro Model for Benchmarking Neuro-Inspired Architectures in Online Learning , 2018, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[3]  Hong Wang,et al.  Loihi: A Neuromorphic Manycore Processor with On-Chip Learning , 2018, IEEE Micro.

[4]  Wayne Luk,et al.  Hardware Design with a Scripting Language , 2003, FPL.

[5]  Joseph McMahan,et al.  A pythonic approach for rapid hardware prototyping and instrumentation , 2017, 2017 27th International Conference on Field Programmable Logic and Applications (FPL).

[6]  Shunning Jiang,et al.  PyOCN: A Unified Framework for Modeling, Testing, and Evaluating On-Chip Networks , 2019, 2019 IEEE 37th International Conference on Computer Design (ICCD).

[7]  Tobias Bjerregaard,et al.  A survey of research and practices of Network-on-chip , 2006, CSUR.

[8]  Johannes Schemmel,et al.  A Mixed-Signal Structured AdEx Neuron for Accelerated Neuromorphic Cores , 2018, IEEE Transactions on Biomedical Circuits and Systems.

[9]  T. Fitzpatrick SystemVerilog for VHDL users , 2004, Proceedings Design, Automation and Test in Europe Conference and Exhibition.

[10]  Todd M. Austin Design for Verification? , 2001, IEEE Des. Test Comput..

[11]  Jan Kuper,et al.  C?aSH: Structural Descriptions of Synchronous Hardware Using Haskell , 2010, 2010 13th Euromicro Conference on Digital System Design: Architectures, Methods and Tools.

[12]  Shunning Jiang,et al.  PyMTL3: A Python Framework for Open-Source Hardware Modeling, Generation, Simulation, and Verification , 2020, IEEE Micro.

[13]  Christopher C. Tison,et al.  Linear programmable nanophotonic processors , 2018, Optica.

[14]  Advait Madhavan,et al.  Streaming Batch Eigenupdates for Hardware Neural Networks , 2019, Front. Neurosci..

[15]  Sandip Ray,et al.  Challenges and Trends in Modern SoC Design Verification , 2017, IEEE Design & Test.

[16]  David A. Patterson,et al.  In-datacenter performance analysis of a tensor processing unit , 2017, 2017 ACM/IEEE 44th Annual International Symposium on Computer Architecture (ISCA).

[17]  Steven J. Plimpton,et al.  Achieving ideal accuracies in analog neuromorphic computing using periodic carry , 2017, 2017 Symposium on VLSI Technology.

[18]  G. W. Burr,et al.  Experimental demonstration and tolerancing of a large-scale neural network (165,000 synapses), using phase-change memory as the synaptic weight element , 2015, 2014 IEEE International Electron Devices Meeting.

[19]  Brian D. Hoskins,et al.  Memory-efficient training with streaming dimensionality reduction , 2020, ArXiv.