Autonomous learning of disparity-vergence behavior through distributed coding and population reward: Basic mechanisms and real-world conditioning on a robot stereo head

A robotic system implementation that exhibits autonomous learning capabilities of effective control for vergence eye movements is presented. The system, directly relying on a distributed (i.e. neural) representation of binocular disparity, shows a large tolerance to the inaccuracies of real stereo heads and to the changeable environment. The proposed approach combines early binocular vision mechanisms with basic learning processes, such as synaptic plasticity and reward modulation. The computational substrate consists of a network of modeled V1 complex cells that act as oriented binocular disparity detectors. The resulting population response, besides implicit binocular depth cues about the environment, also provides a global signal (i.e. the overall activity of the population itself) to describe the state of the system and thus its deviation from the desired vergence position. The proposed network, by taking into account the modification of its internal state as a consequence of the action performed, evolves following a differential Hebbian rule. The overall activity of the population is exploited to derive an intrinsic signal that drives the weights update. Exploiting this signal implies a maximization of the population activity itself, thus providing an highly effective reward for the developing of a stable and accurate vergence behavior. The role of the different orientations in the learning process is evaluated separately against the whole population, evidencing that the interplay among the differently oriented channels allows a faster learning capability and a more accurate control. The efficacy of the proposed intrinsic reward signal is thus comparatively assessed against the ground-truth signal (the actual disparity) providing equivalent results, and thus validating the approach. Trained in a simulated environment, the proposed network, is able to cope with vergent geometry and thus to learn effective vergence movements for static and moving visual targets. Experimental tests with real robot stereo pairs demonstrate the capability of the architecture not just to directly learn from the environment, but to adapt the control to the stimulus characteristics. We implemented a cortical model for the vergence control based on a population of disparity detectors.The model is able to autonomously learn its behavior by means of an internal parameter.The speed of convergence and the precision of the control precision were evaluated on different disparity ranges and learning signals.The informative content of the different orientation channels was assessed.The learning capabilities on real robot stereo pairs demonstrate an adaptation to the stimulus characteristics.

[1]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[2]  R. Leigh,et al.  The neurology of eye movements , 1984 .

[3]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[4]  Manuela Chessa,et al.  A Fast Joint Bioinspired Algorithm for Optic Flow and Two-Dimensional Disparity Estimation , 2009, ICVS.

[5]  Yu Zhao,et al.  Autonomous learning of active multi-scale binocular vision , 2013, 2013 IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL).

[6]  Robert S. Allison,et al.  Coarse-fine dichotomies in human stereopsis , 2009, Vision Research.

[7]  Bertram E. Shi,et al.  Joint development of disparity tuning and vergence control , 2011, 2011 IEEE International Conference on Development and Learning (ICDL).

[8]  J. Daugman Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[9]  B. G. Cumming,et al.  Responses of primary visual cortical neurons to binocular disparity without depth perception , 1997, Nature.

[10]  John L. Semmlow,et al.  Model-based analysis of dynamics in vergence adaptation , 2001, IEEE Trans. Biomed. Eng..

[11]  R Held,et al.  The development of eye alignment, convergence, and sensory binocularity in young infants. , 1994, Investigative ophthalmology & visual science.

[12]  David J. Fleet,et al.  Modelling binocular neurons in the primary visual cortex , 1997 .

[13]  Manuela Chessa,et al.  How a Population-based Representation of Binocular Visual Signal Can Intrinsically Mediate Autonomous Learning of Vergence Control , 2012, INNS-WC.

[14]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[15]  E. Oja Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[16]  R. Held,et al.  Stereoacuity development for crossed and uncrossed disparities in human infants , 1982, Vision Research.

[17]  Gregory C. DeAngelis,et al.  Depth is encoded in the visual cortex by a specialized receptive field structure , 1991, Nature.

[18]  F. A. Miles,et al.  Vergence eye movements in response to binocular disparity without depth perception , 1997, Nature.

[19]  Manuela Chessa,et al.  A cortical model for binocular vergence control without explicit calculation of disparity , 2010, Neurocomputing.

[20]  I. Ohzawa,et al.  Encoding of binocular disparity by complex cells in the cat's visual cortex. , 1996, Journal of neurophysiology.

[21]  Tomaso A. Poggio,et al.  A Canonical Neural Circuit for Cortical Nonlinear Operations , 2008, Neural Computation.

[22]  A. Parker,et al.  Quantitative analysis of the responses of V1 neurons to horizontal disparity in dynamic random-dot stereograms. , 2002, Journal of neurophysiology.

[23]  Manuela Chessa,et al.  A Hierarchical System for a Distributed Representation of the Peripersonal Space of a Humanoid Robot , 2014, IEEE Transactions on Autonomous Mental Development.

[24]  Bertram E. Shi,et al.  Improved Binocular Vergence Control via a Neural Network That Maximizes an Internally Defined Reward , 2011, IEEE Transactions on Autonomous Mental Development.

[25]  Patricia Shaw,et al.  A biologically constrained architecture for developmental learning of eye–head gaze control on a humanoid robot , 2013, Auton. Robots.

[26]  Yu Zhao,et al.  A unified model of the joint development of disparity selectivity and vergence control , 2012, 2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL).

[27]  Manuela Chessa,et al.  A neural model for binocular vergence control without explicit calculation of disparity , 2009, ESANN.

[28]  David J. Fleet,et al.  Neural encoding of binocular disparity: Energy models, position shifts and phase shifts , 1996, Vision Research.

[29]  Bertram E. Shi,et al.  Autonomous Development of Vergence Control Driven by Disparity Energy Neuron Populations , 2010, Neural Computation.

[30]  Manuela Chessa,et al.  Calibrated depth and color cameras for accurate 3D interaction in a stereoscopic augmented reality environment , 2014, J. Vis. Commun. Image Represent..

[31]  Yang Liu,et al.  Disparity statistics in natural scenes. , 2008, Journal of vision.

[32]  Harold E. Bedell,et al.  Pooling signals from vertically and non-vertically orientation-tuned disparity mechanisms in human stereopsis , 2006, Vision Research.

[33]  Fabio Solari,et al.  A compact harmonic code for early vision based on anisotropic frequency channels , 2010, Comput. Vis. Image Underst..

[34]  Zhaoping Li,et al.  Efficient stereo coding in the multiscale representation , 1994 .

[35]  George K. Hung,et al.  A Dual-Mode Dynamic Model of the Vergence Eye Movement System , 1986, IEEE Transactions on Biomedical Engineering.

[36]  Giorgio Cannata,et al.  Implementation of Listing's Law for a Tendon Driven Robot Eye , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[37]  Manuela Chessa,et al.  A neuromorphic control module for real-time vergence eye movements on the iCub robot head , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[38]  Aapo Hyvärinen,et al.  A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images , 2001, Vision Research.

[39]  Wolfgang Maass,et al.  A Reward-Modulated Hebbian Learning Rule Can Explain Experimentally Observed Network Reorganization in a Brain Control Task , 2010, The Journal of Neuroscience.

[40]  Juyang Weng,et al.  Developmental Stereo: Emergence of Disparity Preference in Models of the Visual Cortex , 2009, IEEE Transactions on Autonomous Mental Development.

[41]  H. Öğmen,et al.  PII: S0042-6989(96)00279-9 , 1997 .

[42]  Hanspeter A. Mallot,et al.  Phase-based binocular vergence control and depth reconstruction using active vision , 1994 .

[43]  Patrick D. Roberts,et al.  Computational Consequences of Temporally Asymmetric Learning Rules: I. Differential Hebbian Learning , 1999, Journal of Computational Neuroscience.

[44]  J. Triesch,et al.  Emergence of Disparity Tuning during the Development of Vergence Eye Movements , 2007, 2007 IEEE 6th International Conference on Development and Learning.

[45]  I. Ohzawa,et al.  Stereoscopic depth discrimination in the visual cortex: neurons ideally suited as disparity detectors. , 1990, Science.

[46]  Chuan Yi Tang,et al.  A 2.|E|-Bit Distributed Algorithm for the Directed Euler Trail Problem , 1993, Inf. Process. Lett..

[47]  Jean-Baptiste Durand,et al.  Neural bases of stereopsis across visual field of the alert macaque monkey. , 2007, Cerebral cortex.

[48]  Bertram E. Shi,et al.  The role of orientation diversity in binocular vergence control , 2011, The 2011 International Joint Conference on Neural Networks.

[49]  Radu Horaud,et al.  Cyclopean Geometry of Binocular Vision , 2008, Journal of the Optical Society of America. A, Optics, image science, and vision.

[50]  A. Parker,et al.  Range and mechanism of encoding of horizontal disparity in macaque V1. , 2002, Journal of neurophysiology.

[51]  Agostino Gibaldi,et al.  Learning Eye Vergence Control from a Distributed Disparity Representation , 2010, Int. J. Neural Syst..

[52]  K. Fujii,et al.  Visualization for the analysis of fluid motion , 2005, J. Vis..

[53]  Giorgio Metta,et al.  Design of the robot-cub (iCub) head , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..