Methods for quantifying the informational structure of sensory and motor data

Embodied agents (organisms and robots) are situated in specific environments sampled by their sensors and within which they carry out motor activity. Their control architectures or nervous systems attend to and process streams of sensory stimulation, and ultimately generate sequences of motor actions, which in turn affect the selection of information. Thus, sensory input and motor activity are continuously and dynamically coupled with the surrounding environment. In this article, we propose that the ability of embodied agents to actively structure their sensory input and to generate statistical regularities represents a major functional rationale for the dynamic coupling between sensory and motor systems. Statistical regularities in the multimodal sensory data relayed to the brain are critical for enabling appropriate developmental processes, perceptual categorization, adaptation, and learning. To characterize the informational structure of sensory and motor data, we introduce and illustrate a set of univariate and multivariate statistical measures (available in an accompanying Matlab toolbox). We show how such measures can be used to quantify the information structure in sensory and motor channels of a robot capable of saliency-based attentional behavior, and discuss their potential importance for understanding sensorimotor coordination in organisms and for robot design.

[1]  Jonathan D. Cryer,et al.  Time Series Analysis , 1986 .

[2]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[3]  P. Grassberger.  Finite sample corrections to entropy and dimension estimates , 1988 .

[4]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[5]  M. Kramer.  Nonlinear principal component analysis using autoassociative neural networks , 1991 .

[6]  J. Kurths,et al.  A TEST FOR STATIONARITY: FINDING PARTS IN TIME SERIES APT FOR CORRELATION DIMENSION ESTIMATES , 1993 .

[7]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[8]  D. Ruderman.  The statistics of natural images , 1994 .

[9]  G. Edelman,et al.  A measure for brain complexity: relating functional segregation and integration in the nervous system. , 1994, Proceedings of the National Academy of Sciences of the United States of America.

[10]  W. Bialek,et al.  Naturalistic stimuli increase the rate and efficiency of information transmission by primary auditory afferents , 1995, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[11]  J. Atick,et al.  STATISTICS OF NATURAL TIME-VARYING IMAGES , 1995 .

[12]  D. Lewkowicz,et al.  A dynamic systems approach to the development of cognition and action. , 2007, Journal of cognitive neuroscience.

[13]  J. V. van Hateren,et al.  Modelling the power spectra of natural images: statistics and information. , 1996, Vision research.

[14]  G Tononi,et al.  A complexity measure for selective matching of signals by the brain. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[15]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[16]  J. H. van Hateren,et al.  Modelling the Power Spectra of Natural Images: Statistics and Information , 1996, Vision Research.

[17]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[18]  Thomas Schreiber,et al.  Detecting and Analyzing Nonstationarity in a Time Series Using Nonlinear Cross Predictions , 1997, chao-dyn/9909044.

[19]  G. Edelman,et al.  Complexity and coherency: integrating information in the brain , 1998, Trends in Cognitive Sciences.

[20]  Yasuo Kuniyoshi,et al.  Embedded neural networks: exploiting constraints , 1998, Neural Networks.

[21]  G. Edelman,et al.  Behavioral constraints in the development of neuronal properties: a cortical model embedded in a real-world device. , 1998, Cerebral cortex.

[22]  J. Simonoff.  Smoothing Methods in Statistics , 1998 .

[23]  M. Roulston.  Estimating the errors on measured entropy and mutual information , 1999 .

[24]  Andrew R. Webb,et al.  Statistical Pattern Recognition , 1999 .

[25]  David J. Field,et al.  Wavelets, vision and the statistics of natural scenes , 1999, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[26]  Rolf Pfeifer,et al.  Understanding intelligence , 2020, Inequality by Design.

[27]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[28]  G Tononi,et al.  Theoretical neuroanatomy: relating anatomical and functional connectivity in graphs and cortical connection matrices. , 2000, Cerebral cortex.

[29]  H. Barlow.  The exploitation of regularities in the environment by the brain. , 2001, The Behavioral and brain sciences.

[30]  Eero P. Simoncelli,et al.  Natural image statistics and neural representation. , 2001, Annual review of neuroscience.

[31]  Max Lungarella,et al.  Robots As Cognitive Tools Information theoretic analysis of sensory-motor data , 2001 .

[32]  Stefano Nolfi,et al.  Power and the limits of reactive agents , 2002, Neurocomputing.

[33]  Rolf Pfeifer,et al.  An Optimal Sensor Morphology Improves Adaptability of Neural Network Controllers , 2002, ICANN.

[34]  Thomas P. von Hoff,et al.  Adaptive step-size control in blind source separation , 2002, Neurocomputing.

[35]  Olaf Sporns,et al.  An Embodied Model of Learning, Plasticity, and Reward , 2002, Adapt. Behav..

[36]  Chen Yu,et al.  A multimodal learning interface for grounding spoken language in sensory perceptions , 2003, ICMI '03.

[37]  Antonio Torralba,et al.  Statistics of natural image categories , 2003, Network.

[38]  Olaf Sporns,et al.  Information-Theoretical Aspects of Embodied Artificial Intelligence , 2003, Embodied Artificial Intelligence.

[39]  O. Spoms,et al.  Generating structure in sensory data through coordinated motor activity , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[40]  Konrad P. Körding,et al.  The world from a cat’s perspective – statistics of natural videos , 2003, Biological Cybernetics.

[41]  Giorgio Metta,et al.  Early integration of vision and manipulation , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[42]  M. Lewicki,et al.  Learning higher-order structures in natural images , 2003, Network.

[43]  Giorgio Metta,et al.  Better Vision through Manipulation , 2003, Adapt. Behav..

[44]  I. René J. A. te Boekhorst,et al.  Dimensionality Reduction through Sensory-Motor Coordination , 2003, ICANN.

[45]  Anil K. Seth,et al.  Environment and Behavior Influence the Complexity of Evolved Neural Networks , 2004, Adapt. Behav..

[46]  Chrystopher L. Nehaniv,et al.  The Effects on Visual Information in a Robot in Environments with Oriented Contours , 2004 .

[47]  A. Seth.  Causal connectivity of evolved neural networks during behavior. , 2005, Network.

[48]  M. Lungarella,et al.  Information Self-Structuring: Key Principle for Learning and Development , 2005, Proceedings. The 4nd International Conference on Development and Learning, 2005..

[49]  Danesh Tarapore,et al.  Quantifying patterns of agent-environment interaction , 2006, Robotics Auton. Syst..

[50]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[51]  Jeffrey L. Krichmar,et al.  Spatial navigation and causal analysis in a brain-based device modeling cortical-hippocampal interactions , 2007, Neuroinformatics.

[52]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .