Learning Data Representations with Sparse Coding Neural Gas

We consider the problem of learning an unknown (overcom- plete) basis from an unknown sparse linear combination. Introducing the "sparse coding neural gas" algorithm, we show how to employ a combina- tion of the original neural gas algorithm and Oja's rule in order to learn a simple sparse code that represents each training sample by a multiple of one basis vector. We generalise this algorithm using orthogonal matching pursuit in order to learn a sparse code where each training sample is rep- resented by a linear combination of k basis elements. We show that this method can be used to learn artificial sparse overcomplete codes.