Some Biases for EfficientLearning of Spatial, Temporal, and Spatio-Temporal Patterns

This paper introduces and explores some representational biases for efficient learning of spatial, temporal, or spatio-temporal patterns in connectionist networks (CN) - massively parallel networks of simple computing elements. It examines learning mechanisms that constructively build up network structures that encode information from environmental stimuli at successively higher resolutions as needed for the tasks (e.g., perceptual recognition) that the network has to perform. Some simple examples are presented to illustrate the the basic structures and processes used in such networks to ensure the parsimony of learned representations - by guiding the system to focus its efforts at the minimal adequate resolution. Several extensions of the basic algorithm for efficient learning using multi-resolution representations of spatial, temporal, or spatio-temporal patterns are discussed. 1. Multi-Resolution Iconic Representations Environmental stimuli (e.g., 2-dimensional visual images) typically contain features over multiple scales. Multiresolution pattern encodings provide a basis for analyzing features in the environmental stimuli at different scales (Uhr, 1972; Rosenfeld, 1984; Dyer, 1987). This section introduces multi-resolution representations and their use in efficientlearning of spatial, temporal, and spatio-temporal patterns. Typically, a multi-resolution encoding scheme transforms the input (e.g., a 2-dimensional image) into a set of maps at successively coarser resolutions, each making explicit image features at a specificscale. An example of such a scheme is the gaussian pyramid in which successively higher levels encode blurred and sub-sampled versions of the immediately lower level (where the blurring may be applied to any local property of the stimulus e.g., intensity, color, texture, etc). If the stimulus is a 512x512 image, blurring with a 2-dimensional gaussian kernel g (x ,y ) = 2πγ 2