Distribution Approximation : An In-Place Developmental Algorithm for Sensory Cortices and A Hypothesis

Orientation selective cells in V1 are well known, but the underlying computational principles that guide their emergence (i.e., development) are still illusive. The result reported here is based on high-dimensional probability distribution using a network structure. We introduce a concept called lobe component. Each lobe component represents a high concentration of probability density of the high-dimensional sensory inputs. A developmental algorithm for sensory network has been designed and tested based on the concept of lobe components. Our simulation result of network development revealed that many lobe component cells developed by the algorithm from natural images are orientational selective cells similar to those found in V1. Therefore, we make a hypothesis that orientation selection is only a natural consequence of cortical development but probability density estimation by neurons is a fundamental principle of cortical development. If this hypothesis is true, the principle of neuronal probability approximation may have deep implications to the development and self-organization of other sensory cortices (e.g., auditory and somatosensory cortices) and higher cortices. The proposed developmental algorithm is not meant to be biologically verified but is biological plausible. It is based on two well-known computational neural mechanisms, Hebbian learning and lateral inhibition. It is an in-place developmental algorithm in which every cell is both a signal processor and the developer of the cortex. There is no need for a separate network to deal with the development (and learning). The algorithm is purely incremental in the sense that there is no need for extra storage for information other than that can be stored and incrementally computed by the processing network itself. To clarify technically, four types of algorithms have been considered with progressively more restrictive conditions: batch (Type-1), block-incremental (Type-2), incremental (Type-3) and covariance-free incremental (Type-4). The proposed Candid Covariance-free Incremental (CCI) Lobe Component Analysis (LCA) algorithm seems the first Type-4 algorithm for independent component analysis (ICA). The preliminary simulation studies showed that it outperformed, in convergence rate and computation time, some well-known state-of-the-art ICA algorithms in Type-3 through Type 1 for high-dimensional data streams. The proposed LCA algorithm also has a simple structure, with the lowest possible space and time complexities.