High-Dimensional Computing as a Nanoscalable Paradigm

We outline a model of computing with high-dimensional (HD) vectors—where the dimensionality is in the thousands. It is built on ideas from traditional (symbolic) computing and artificial neural nets/deep learning, and complements them with ideas from probability theory, statistics, and abstract algebra. Key properties of HD computing include a well-defined set of arithmetic operations on vectors, generality, scalability, robustness, fast learning, and ubiquitous parallel operation, making it possible to develop efficient algorithms for large-scale real-world tasks. We present a 2-D architecture and demonstrate its functionality with examples from text analysis, pattern recognition, and biosignal processing, while achieving high levels of classification accuracy (close to or above conventional machine-learning methods), energy efficiency, and robustness with simple algorithms that learn fast. HD computing is ideally suited for 3-D nanometer circuit technology, vastly increasing circuit density and energy efficiency, and paving a way to systems capable of advanced cognitive tasks.

[1]  Okko Johannes Räsänen,et al.  Generating Hyperdimensional Distributed Representations from Continuous-Valued Multivariate Sensory Input , 2015, CogSci.

[2]  Christian Biemann,et al.  Corpus Portal for Search in Monolingual Corpora , 2006, LREC.

[3]  Earl E. Swartzlander,et al.  Bidirectional voltage biased implication operations using SiOx based unipolar memristors , 2015 .

[4]  Thorsten Joachims,et al.  Text Categorization with Support Vector Machines: Learning with Many Relevant Features , 1998, ECML.

[5]  W. Marsden.  I and J , 2012 .

[6]  Okko Johannes Räsänen,et al.  Modeling Dependencies in Multiple Parallel Data Streams with Hyperdimensional Computing , 2014, IEEE Signal Processing Letters.

[7]  Stephen I. Gallant,et al.  Representing Objects, Relations, and Sequences , 2013, Neural Computation.

[8]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[9]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Li Ji,et al.  Integrated one diode-one resistor architecture in nanopillar SiOx resistive switching memory by nanosphere lithography. , 2014, Nano letters.

[11]  Luca Benini,et al.  PULP: A parallel ultra low power platform for next generation IoT applications , 2015, 2015 IEEE Hot Chips 27 Symposium (HCS).

[12]  Jan M. Rabaey,et al.  A Robust and Energy-Efficient Classifier Using Brain-Inspired Hyperdimensional Computing , 2016, ISLPED.

[13]  Ross W. Gayler,et al.  Multiplicative Binding, Representation Operators & Analogy , 1998 .

[14]  Evgeny Osipov,et al.  Recognizing Permuted Words with Vector Symbolic Architectures: A Cambridge Test for Machines , 2016, BICA.

[15]  Okko Johannes Räsänen,et al.  Sequence Prediction With Sparse Distributed Hyperdimensional Coding Applied to the Analysis of Mobile Phone Use Patterns , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Y. Ahmet Sekercioglu,et al.  Holographic Graph Neuron: A Bioinspired Architecture for Pattern Processing , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[17]  Tony A. Plate,et al.  Holographic reduced representations , 1995, IEEE Trans. Neural Networks.

[18]  P. Kanerva.  Fully Distributed Representation , 1997 .

[19]  Anders Holst,et al.  Random indexing of text samples for latent semantic analysis , 2000 .

[20]  RecchiaGabriel,et al.  Encoding sequential information in semantic space models , 2015 .

[21]  Luca Benini,et al.  Hyperdimensional biosignal processing: A case study for EMG-based hand gesture recognition , 2016, 2016 IEEE International Conference on Rebooting Computing (ICRC).

[22]  Philipp Koehn,et al.  Europarl: A Parallel Corpus for Statistical Machine Translation , 2005, MTSUMMIT.

[23]  Ross W. Gayler.  Vector Symbolic Architectures answer Jackendoff's challenges for cognitive neuroscience , 2004, ArXiv.

[24]  Jung-Hui Chen,et al.  Charge Quantity Influence on Resistance Switching Characteristic During Forming Process , 2013, IEEE Electron Device Letters.

[25]  O. Richard,et al.  Vacancy-modulated conductive oxide resistive RAM (VMCO-RRAM): An area-scalable switching current, self-compliant, highly nonlinear and wide on/off-window resistive switching cell , 2013, 2013 IEEE International Electron Devices Meeting.

[26]  Pentti Kanerva,et al.  Hyperdimensional Computing: An Introduction to Computing in Distributed Representation with High-Dimensional Random Vectors , 2009, Cognitive Computation.

[27]  Evgeny Osipov,et al.  On Bidirectional Transitions between Localist and Distributed Representations: The Case of Common Substrings Search Using Vector Symbolic Architecture , 2014, BICA.

[28]  Luca Benini,et al.  A Versatile Embedded Platform for EMG Acquisition and Gesture Recognition , 2015, IEEE Transactions on Biomedical Circuits and Systems.

[29]  Shimeng Yu,et al.  HfOx based vertical resistive random access memory for cost-effective 3D cross-point architecture without cell selector , 2012, 2012 International Electron Devices Meeting.

[30]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[31]  S. Burc Eryilmaz,et al.  Four-layer 3D vertical RRAM integrated with FinFET as a versatile computing unit for brain-inspired cognitive information processing , 2016, 2016 IEEE Symposium on VLSI Technology.

[32]  Pentti Kanerva,et al.  Binary Spatter-Coding of Ordered K-Tuples , 1996, ICANN.

[33]  Yen-Ting Chen,et al.  Understanding the resistive switching characteristics and mechanism in active SiOx-based resistive switching memory , 2012 .

[34]  Dmitri A. Rachkovskij,et al.  Representation and Processing of Structures with Binary Sparse Distributed Codes , 2001, IEEE Trans. Knowl. Data Eng..

[35]  Nikolaos Papakonstantinou,et al.  Fault detection in the hyperspace: Towards intelligent automation systems , 2015, 2015 IEEE 13th International Conference on Industrial Informatics (INDIN).

[36]  Jan M. Rabaey,et al.  Hyperdimensional computing with 3D VRRAM in-memory kernels: Device-architecture co-design for energy-efficient, error-resilient language recognition , 2016, 2016 IEEE International Electron Devices Meeting (IEDM).

[37]  Bart De Moor,et al.  Geometric Analogue of Holographic Reduced Representation , 2007, ArXiv.

[38]  P. Kanerva,et al.  Hyperdimensional Computing for Text Classification , 2016 .

[39]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[40]  Hong-Yu Chen,et al.  Metal oxide-resistive memory using graphene-edge electrodes , 2015, Nature Communications.

[41]  Shimeng Yu,et al.  Metal–Oxide RRAM , 2012, Proceedings of the IEEE.

[42]  Evgeny Osipov,et al.  Brain-like classifier of temporal patterns , 2014, 2014 International Conference on Computer and Information Sciences (ICCOINS).

[43]  Fei Zhou,et al.  Stabilization of multiple resistance levels by current-sweep in SiOx-based resistive switching memory , 2015 .

[44]  Jan M. Rabaey,et al.  Hyperdimensional computing for noninvasive brain-computer interfaces: Blind and one-shot classification of EEG error-related potentials , 2017 .

[45]  Amritesh Rai,et al.  Characteristics and mechanism study of cerium oxide based random access memories , 2015 .

[46]  Aditya Joshi,et al.  Language Geometry Using Random Indexing , 2016, QI.

[47]  Michael N. Jones,et al.  Encoding Sequential Information in Semantic Space Models: Comparing Holographic Reduced Representation and Random Permutation , 2015, Comput. Intell. Neurosci..