Using Evolutionary Algorithms for Fitting High-Dimensional Models to Neuronal Data

In the study of neurosciences, and of complex biological systems in general, there is frequently a need to fit mathematical models with large numbers of parameters to highly complex datasets. Here we consider algorithms of two different classes, gradient following (GF) methods and evolutionary algorithms (EA) and examine their performance in fitting a 9-parameter model of a filter-based visual neuron to real data recorded from a sample of 107 neurons in macaque primary visual cortex (V1). Although the GF method converged very rapidly on a solution, it was highly susceptible to the effects of local minima in the error surface and produced relatively poor fits unless the initial estimates of the parameters were already very good. Conversely, although the EA required many more iterations of evaluating the model neuron’s response to a series of stimuli, it ultimately found better solutions in nearly all cases and its performance was independent of the starting parameters of the model. Thus, although the fitting process was lengthy in terms of processing time, the relative lack of human intervention in the evolutionary algorithm, and its ability ultimately to generate model fits that could be trusted as being close to optimal, made it far superior in this particular application than the gradient following methods. This is likely to be the case in many further complex systems, as are often found in neuroscience.

[1]  J. Movshon,et al.  Nature and interaction of signals from the receptive field center and surround in macaque V1 neurons. , 2002, Journal of neurophysiology.

[2]  Eric Jones,et al.  SciPy: Open Source Scientific Tools for Python , 2001 .

[3]  P. Lennie,et al.  Early and Late Mechanisms of Surround Suppression in Striate Cortex of Macaque , 2005, The Journal of Neuroscience.

[4]  J. Daugman Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[5]  Nikolaus Hansen,et al.  On the Adaptation of Arbitrary Normal Mutation Distributions in Evolution Strategies: The Generating Set Adaptation , 1995, ICGA.

[6]  N. Hansen,et al.  Convergence Properties of Evolution Strategies with the Derandomized Covariance Matrix Adaptation: T , 1997 .

[7]  J. Movshon,et al.  Linearity and Normalization in Simple Cells of the Macaque Primary Visual Cortex , 1997, The Journal of Neuroscience.

[8]  S Marcelja,et al.  Mathematical description of the responses of simple cortical cells. , 1980, Journal of the Optical Society of America.

[9]  William E. Sabin The Hilbert Transform , 2007 .

[10]  Romain Brette,et al.  Neuroinformatics Original Research Article Brian: a Simulator for Spiking Neural Networks in Python , 2022 .

[11]  D. Heeger Normalization of cell responses in cat striate cortex , 1992, Visual Neuroscience.

[12]  Timothy J. Blanche,et al.  Python for Large-Scale Electrophysiology , 2009, Front. Neuroinform..

[13]  Lothar Thiele,et al.  Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach , 1999, IEEE Trans. Evol. Comput..

[14]  Olli Yli-Harja,et al.  Comparison of automated parameter estimation methods for neuronal signaling networks , 2006, Neurocomputing.

[15]  Gaute T. Einevoll,et al.  Sharing with Python , 2009, Front. Neurosci..

[16]  M. Carandini,et al.  Summation and division by neurons in primate visual cortex. , 1994, Science.

[17]  Jeffrey C. Lagarias,et al.  Convergence Properties of the Nelder-Mead Simplex Method in Low Dimensions , 1998, SIAM J. Optim..

[18]  Richard J Krauzlis,et al.  Spatial integration by MT pattern neurons: a closer look at pattern-to-component effects and the role of speed tuning. , 2008, Journal of vision.

[19]  Pierre Yger,et al.  PyNN: A Common Interface for Neuronal Network Simulators , 2008, Front. Neuroinform..

[20]  Jim Smith,et al.  Memetic Algorithms: The Polynomial Local Search Complexity Theory Perspective , 2008, J. Math. Model. Algorithms.

[21]  Tamar Schlick,et al.  TNPACK—A truncated Newton minimization package for large-scale problems: I. Algorithm and usage , 1992, TOMS.

[22]  Michael T. Heath,et al.  Scientific Computing: An Introductory Survey , 1996 .

[23]  A J Ahumada,et al.  Model of human visual-motion sensing. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[24]  C. G. Broyden Quasi-Newton methods and their application to function minimisation , 1967 .

[25]  Kwang-Hyun Cho,et al.  Inferring gene regulatory networks from temporal expression profiles under time-delay and noise , 2007, Comput. Biol. Chem..

[26]  Örjan Ekeberg,et al.  Large-Scale Modeling – a Tool for Conquering the Complexity of the Brain , 2008, Frontiers Neuroinformatics.

[27]  Marco Laumanns,et al.  Performance assessment of multiobjective optimizers: an analysis and review , 2003, IEEE Trans. Evol. Comput..

[28]  J. Nocedal Updating Quasi-Newton Matrices With Limited Storage , 1980 .

[29]  Jasper A Vrugt,et al.  Improved evolutionary optimization from genetically adaptive multimethod search , 2007, Proceedings of the National Academy of Sciences.

[30]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[31]  P. Lennie,et al.  Chromatic Gain Controls in Visual Cortical Neurons , 2005, The Journal of Neuroscience.

[32]  S. Metlapally,et al.  The effect of positive lens defocus on ocular growth and emmetropization in the tree shrew. , 2008, Journal of vision.

[33]  Kalyanmoy Deb,et al.  Multi-objective Genetic Algorithms: Problem Difficulties and Construction of Test Problems , 1999, Evolutionary Computation.

[34]  Albert A. Groenwold,et al.  Multiple Parallel Local Searches in Global Optimization , 2000, PVM/MPI.

[35]  Jonathan W Peirce,et al.  Cortical representation of color is binocular. , 2008, Journal of vision.

[36]  Alexei N. Pankratov,et al.  A semiempirical quantum chemical testing of thermodynamic and molecular properties of arsenic compounds , 2000 .

[37]  L. C W. Dixon,et al.  Quasi-newton algorithms generate identical points , 1972, Math. Program..

[38]  Henry Markram,et al.  A Novel Multiple Objective Optimization Framework for Constraining Conductance-Based Neuron Models by Experimental Data , 2007, Front. Neurosci..

[39]  Stefano Fanelli,et al.  Matrix algebras in Quasi-Newton methods for unconstrained minimization , 2003, Numerische Mathematik.

[40]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[41]  S. Siegel,et al.  Nonparametric Statistics for the Behavioral Sciences , 2022, The SAGE Encyclopedia of Research Design.

[42]  Douglas B. Kell,et al.  Non-linear optimization of biochemical pathways: applications to metabolic engineering and parameter estimation , 1998, Bioinform..

[43]  Erik De Schutter,et al.  Automated neuron model optimization techniques: a review , 2008, Biological Cybernetics.

[44]  M. Hazewinkel Encyclopaedia of mathematics , 1987 .

[45]  Ivo F. Sbalzariniy,et al.  Multiobjective optimization using evolutionary algorithms , 2000 .

[46]  Jijun Zhao,et al.  Gas molecule adsorption in carbon nanotubes and nanotube bundles , 2002 .

[47]  Eero P. Simoncelli,et al.  Modeling Surround Suppression in V1 Neurons with a Statistically Derived Normalization Model , 1998, NIPS.

[48]  M. J. Box A Comparison of Several Current Optimization Methods, and the use of Transformations in Constrained Problems , 1966, Comput. J..

[49]  J. Kent Information gain and a general measure of correlation , 1983 .

[50]  Jorge Nocedal,et al.  A Numerical Study of the Limited Memory BFGS Method and the Truncated-Newton Method for Large Scale Optimization , 1991, SIAM J. Optim..

[51]  D. C. Howell Fundamental Statistics for the Behavioral Sciences , 1985 .

[52]  R. Shapley,et al.  Visual spatial characterization of macaque V1 neurons. , 2001, Journal of neurophysiology.