Adaptive sampling by information maximization.

The investigation of input-output systems often requires a sophisticated choice of test inputs to make the best use of limited experimental time. Here we present an iterative algorithm that continuously adjusts an ensemble of test inputs on-line, subject to the data already acquired about the system under study. The algorithm focuses the input ensemble by maximizing the mutual information between input and output. We apply the algorithm to simulated neurophysiological experiments and show that it serves to extract the ensemble of stimuli that a given neural system "expects" as a result of its natural history.

[1]  William Bialek,et al.  Entropy and Information in Neural Spike Trains , 1996, cond-mat/9603127.

[2]  E Harth,et al.  Alopex: a stochastic method for determining visual receptive fields. , 1974, Vision research.

[3]  Y. W. Lee,et al.  Measurement of the Wiener Kernels of a Non-linear System by Cross-correlation† , 1965 .

[4]  Claude E. Shannon,et al.  A mathematical theory of communication , 1948, MOCO.

[5]  W. Bialek,et al.  Naturalistic stimuli increase the rate and efficiency of information transmission by primary auditory afferents , 1995, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[6]  D. Vernon.  Inform , 1995, Encyclopedia of the UN Sustainable Development Goals.

[7]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[8]  Eugene M. Izhikevich,et al.  Neural excitability, Spiking and bursting , 2000, Int. J. Bifurc. Chaos.

[9]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[10]  G. Buzsáki,et al.  Gamma Oscillation by Synaptic Inhibition in a Hippocampal Interneuronal Network Model , 1996, The Journal of Neuroscience.

[11]  Richard E. Blahut,et al.  Computation of channel capacity and rate-distortion functions , 1972, IEEE Trans. Inf. Theory.

[12]  N. Wiener,et al.  Nonlinear Problems in Random Theory , 1964 .

[13]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[14]  T. Poggio,et al.  Stochastic Identification Methods for Nonlinear Systems: An Extension of the Wiener Theory , 1978 .

[15]  I. Nelken,et al.  In search of the best stimulus: An optimization procedure for finding efficient stimuli in the cat auditory cortex , 1994, Hearing Research.

[16]  Suguru Arimoto,et al.  An algorithm for computing the capacity of arbitrary discrete memoryless channels , 1972, IEEE Trans. Inf. Theory.

[17]  C. Itzykson,et al.  Statistical Field Theory: Random geometry , 1989 .

[18]  Christian K. Machens,et al.  Representation of Acoustic Communication Signals by Insect Auditory Receptor Neurons , 2001, The Journal of Neuroscience.