Deep learning incorporating biologically inspired neural dynamics and in-memory computing

[1]  Hesham Mostafa,et al.  Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-based optimization to spiking neural networks , 2019, IEEE Signal Processing Magazine.

[2]  Romain Brette,et al.  Brian 2: an intuitive and efficient neural simulator , 2019, bioRxiv.

[3]  Meng-Fan Chang,et al.  24.1 A 1Mb Multibit ReRAM Computing-In-Memory Macro with 14.6ns Parallel MAC Computing Time for CNN Based AI Edge Processors , 2019, 2019 IEEE International Solid- State Circuits Conference - (ISSCC).

[4]  Emre Neftci,et al.  Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-based optimization to spiking neural networks , 2019, IEEE Signal Processing Magazine.

[5]  Anthony Maida,et al.  BP-STDP: Approximating Backpropagation using Spike Timing Dependent Plasticity , 2017, Neurocomputing.

[6]  Evangelos S. Eleftheriou,et al.  Deep learning incorporating biologically-inspired neural dynamics , 2018 .

[7]  Michael Pfeiffer,et al.  Deep Learning With Spiking Neurons: Opportunities and Challenges , 2018, Front. Neurosci..

[8]  E. Eleftheriou,et al.  A phase-change memory model for neuromorphic computing , 2018, Journal of Applied Physics.

[9]  Abu Sebastian,et al.  Tutorial: Brain-inspired computing using phase-change memory devices , 2018, Journal of Applied Physics.

[10]  Heiner Giefers,et al.  Compressed Sensing With Approximate Message Passing Using In-Memory Computing , 2018, IEEE Transactions on Electron Devices.

[11]  Gopalakrishnan Srinivasan,et al.  Training Deep Spiking Convolutional Neural Networks With STDP-Based Unsupervised Pre-training Followed by Supervised Fine-Tuning , 2018, Front. Neurosci..

[12]  Bipin Rajendran,et al.  Spiking neural networks for handwritten digit recognition - Supervised learning and network optimization , 2018, Neural Networks.

[13]  Ryutaro Yasuhara,et al.  A 4M Synapses integrated Analog ReRAM based 66.5 TOPS/W Neural-Network Processor with Cell Current Controlled Writing and Flexible Network Architecture , 2018, 2018 IEEE Symposium on VLSI Technology.

[14]  Robert A. Legenstein,et al.  Long short-term memory and Learning-to-learn in networks of spiking neurons , 2018, NeurIPS.

[15]  Meng-Fan Chang,et al.  A 65nm 1Mb nonvolatile computing-in-memory ReRAM macro with sub-16ns multiply-and-accumulate for binary DNN AI edge processors , 2018, 2018 IEEE International Solid - State Circuits Conference - (ISSCC).

[16]  Hong Wang,et al.  Loihi: A Neuromorphic Manycore Processor with On-Chip Learning , 2018, IEEE Micro.

[17]  Terrence J. Sejnowski,et al.  Gradient Descent for Spiking Neural Networks , 2017, NeurIPS.

[18]  Lei Deng,et al.  Spatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks , 2017, Front. Neurosci..

[19]  Evangelos Eleftheriou,et al.  Mixed-precision architecture based on computational memory for training deep neural networks , 2018, 2018 IEEE International Symposium on Circuits and Systems (ISCAS).

[20]  Qinru Qiu,et al.  A spike-based long short-term memory on a neurosynaptic processor , 2017, 2017 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[21]  Yusuf Leblebici,et al.  Unsupervised Learning Using Phase-Change Synapses and Complementary Patterns , 2017, ICANN.

[22]  Yusuf Leblebici,et al.  Neuromorphic system with phase-change synapses for pattern learning and feature extraction , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[23]  Evangelos Eleftheriou,et al.  Fatiguing STDP: Learning from spike-timing codes in the presence of rate codes , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[24]  Yoshua Bengio,et al.  STDP-Compatible Approximation of Backpropagation in an Energy-Based Model , 2017, Neural Computation.

[25]  Pritish Narayanan,et al.  Neuromorphic computing using non-volatile memory , 2017 .

[26]  Lior Wolf,et al.  Using the Output Embedding to Improve Language Models , 2016, EACL.

[27]  Jürgen Schmidhuber,et al.  LSTM: A Search Space Odyssey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[28]  Tobi Delbrück,et al.  Training Deep Spiking Neural Networks Using Backpropagation , 2016, Front. Neurosci..

[29]  Manuel Le Gallo,et al.  Stochastic phase-change neurons. , 2016, Nature nanotechnology.

[30]  E. Eleftheriou,et al.  All-memristive neuromorphic computing with level-tuned neurons , 2016, Nanotechnology.

[31]  Evangelos Eleftheriou,et al.  Detecting Correlations Using Phase-Change Neurons and Synapses , 2016, IEEE Electron Device Letters.

[32]  Evangelos Eleftheriou,et al.  Learning spatio-temporal patterns in the presence of input noise using phase-change memristors , 2016, 2016 IEEE International Symposium on Circuits and Systems (ISCAS).

[33]  Andrew S. Cassidy,et al.  Convolutional networks for fast, energy-efficient neuromorphic computing , 2016, Proceedings of the National Academy of Sciences.

[34]  Paolo Fantini,et al.  Unsupervised Learning by Spike Timing Dependent Plasticity in Phase Change Memory (PCM) Synapses , 2016, Front. Neurosci..

[35]  Andrew S. Cassidy,et al.  Conversion of artificial recurrent neural networks to spiking neural networks for low-power neuromorphic hardware , 2016, 2016 IEEE International Conference on Rebooting Computing (ICRC).

[36]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Chong Wang,et al.  Deep Speech 2 : End-to-End Speech Recognition in English and Mandarin , 2015, ICML.

[38]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Timothée Masquelier,et al.  Bio-inspired unsupervised learning of visual features leads to robust invariant object recognition , 2015, Neurocomputing.

[40]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[41]  Karlheinz Meier,et al.  A mixed-signal universal neuromorphic computing system , 2015, 2015 IEEE International Electron Devices Meeting (IEDM).

[42]  Kendra S. Burbank.  Mirrored STDP Implements Autoencoder Learning in a Network of Spiking Neurons , 2015, PLoS Comput. Biol..

[43]  Chris Eliasmith,et al.  Spiking Deep Networks with LIF Neurons , 2015, ArXiv.

[44]  Matthew Cook,et al.  Unsupervised learning of digit recognition using spike-timing-dependent plasticity , 2015, Front. Comput. Neurosci..

[45]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[46]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  G. W. Burr,et al.  Experimental demonstration and tolerancing of a large-scale neural network (165,000 synapses), using phase-change memory as the synaptic weight element , 2015, 2014 IEEE International Electron Devices Meeting.

[48]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[49]  Andrew S. Cassidy,et al.  Real-Time Scalable Cortical Computing at 46 Giga-Synaptic OPS/Watt with ~100× Speedup in Time-to-Solution and ~100,000× Reduction in Energy-to-Solution , 2014, SC14: International Conference for High Performance Computing, Networking, Storage and Analysis.

[50]  Wulfram Gerstner,et al.  Neuronal Dynamics: From Single Neurons To Networks And Models Of Cognition , 2014 .

[51]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[52]  Wojciech Zaremba,et al.  Recurrent Neural Network Regularization , 2014, ArXiv.

[53]  Yoshua Bengio,et al.  On the Properties of Neural Machine Translation: Encoder–Decoder Approaches , 2014, SSST@EMNLP.

[54]  Rodrigo Alvarez-Icaza,et al.  Neurogrid: A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations , 2014, Proceedings of the IEEE.

[55]  Chris Eliasmith,et al.  A spiking neural model applied to the study of human performance and cognitive decline on Raven's Advanced , 2014 .

[56]  Tobi Delbruck,et al.  Real-time classification and sensor fusion with a spiking deep belief network , 2013, Front. Neurosci..

[57]  Yoshua Bengio,et al.  Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation , 2013, ArXiv.

[58]  Chris Eliasmith,et al.  How to Build a Brain: A Neural Architecture for Biological Cognition , 2013 .

[59]  Andrew L. Maas.  Rectifier Nonlinearities Improve Neural Network Acoustic Models , 2013 .

[60]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[61]  Trevor Bekolay,et al.  A Large-Scale Model of the Functioning Brain , 2012, Science.

[62]  Damien Querlioz,et al.  Extraction of temporally correlated features from dynamic vision sensors with spike-timing-dependent plasticity , 2012, Neural Networks.

[63]  Yoshua Bengio,et al.  Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription , 2012, ICML.

[64]  Byoungil Lee,et al.  Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. , 2012, Nano letters.

[65]  Damien Querlioz,et al.  Simulation of a memristor-based spiking neural network immune to device variations , 2011, The 2011 International Joint Conference on Neural Networks.

[66]  Lukás Burget,et al.  Extensions of recurrent neural network language model , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[67]  Haralampos Pozidis,et al.  Programming algorithms for multilevel phase-change memory , 2011, 2011 IEEE International Symposium of Circuits and Systems (ISCAS).

[68]  C. Hagleitner,et al.  Device, circuit and system-level analysis of noise in multi-bit phase-change memory , 2010, 2010 International Electron Devices Meeting.

[69]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[70]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[71]  Patrice Y. Simard,et al.  Best practices for convolutional neural networks applied to visual document analysis , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[72]  Haim Sompolinsky,et al.  Learning Input Correlations through Nonlinear Temporally Asymmetric Hebbian Plasticity , 2003, The Journal of Neuroscience.

[73]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[74]  Sander M. Bohte,et al.  Error-backpropagation in temporally encoded networks of spiking neurons , 2000, Neurocomputing.

[75]  Peter Dayan,et al.  Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems , 2001 .

[76]  Wolfgang Maass,et al.  On the Computational Power of Winner-Take-All , 2000, Neural Computation.

[77]  Jürgen Schmidhuber,et al.  Learning to Forget: Continual Prediction with LSTM , 2000, Neural Computation.

[78]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[79]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[80]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[81]  H. Markram,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997, Science.

[82]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[83]  Carver A. Mead,et al.  Neuromorphic electronic systems , 1990, Proc. IEEE.

[84]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1990, Bulletin of mathematical biology.

[85]  PAUL J. WERBOS,et al.  Generalization of backpropagation with application to a recurrent gas market model , 1988, Neural Networks.