Hyperdimensional Computing: An Introduction to Computing in Distributed Representation with High-Dimensional Random Vectors
The 1990s saw the emergence of cognitive models that depend on very high dimensionality and randomness. They include Holographic Reduced Representations, Spatter Code, Semantic Vectors, Latent Semantic Analysis, Context-Dependent Thinning, and Vector-Symbolic Architecture. They represent things in high-dimensional vectors that are manipulated by operations that produce new high-dimensional vectors in the style of traditional computing, in what is called here hyperdimensional computing on account of the very high dimensionality. The paper presents the main ideas behind these models, written as a tutorial essay in hopes of making the ideas accessible and even provocative. A sketch of how we have arrived at these models, with references and pointers to further reading, is given at the end. The thesis of the paper is that hyperdimensional representation has much to offer to students of cognitive science, theoretical neuroscience, computer science and engineering, and mathematics.
Nonnegative Matrix Factorization and Probabilistic Latent Semantic Indexing: Equivalence Chi-Square Statistic, and a Hybrid Method
Non-negative Matrix Factorization (NMF) and Probabilistic Latent Semantic Indexing (PLSI) have been successfully applied to document clustering recently. In this paper, we show that PLSI and NMF optimize the same objective function, although PLSI and NMF are different algorithms as verified by experiments. This provides a theoretical basis for a new hybrid method that runs PLSI and NMF alternatively, each jumping out of local minima of the other method successively, thus achieving better final solution. Extensive experiments on 5 real-life datasets show relations between NMF and PLSI, and indicate the hybrid method lead to significant improvements over NMF-only or PLSI-only methods. We also show that at first order approximation, NMF is identical to χ2-statistic.
neural network power system internet of things electric vehicle data analysi renewable energy smart grid learning algorithm power grid image compression hyperspectral image matrix factorization source separation cyber-physical system energy management system sparse representation deep convolutional cloud storage blind source separation demand response blind source gradient method renewable energy system grid system dictionary learning hyperspectral datum latent semantic spectral clustering nonnegative matrix nonnegative matrix factorization hyperspectral imagery low rank image representation image inpainting public cloud matrix completion spectral datum smart grid system smart grid technology remote datum smart grid communication tensor factorization data matrix latent factor future smart grid factorization method spectral unmixing grid communication hyperspectral unmixing international system future smart smart power grid nonnegative matrice power grid system dictionary learning algorithm matrix factorization method data possession projected gradient graph regularized factorization based nonnegative tensor provable data possession system of units image inpainting method smart grid security provable datum public cloud storage matrix factorization technique projected gradient method factorization technique nonnegative tensor factorization nmf algorithm low-rank matrix factorization exemplar-based image inpainting image inpainting technique emerging smart grid matrix factorization problem multiplicative update based image inpainting regularized nonnegative matrix constrained nonnegative matrix sparse nonnegative kernel k-means clustering regularized nonnegative sparse nonnegative matrix matrix and tensor sparse nmf constrained nonnegative high-dimensional vector nmf method orthogonal nonnegative matrix graph regularized nonnegative nonnegative datum multi-way datum nonnegative tucker decomposition lee and seung weighted nonnegative matrix weighted nonnegative robust nonnegative matrix projective nonnegative matrix als algorithm robust nonnegative input data matrix projective nonnegative semantic image inpainting fast nonnegative wind power