MASSACHUSETTS INSTITUTE OF TECHNOLOGY ARTIFICIAL INTELLIGENCE LABORATORY and CENTER FOR BIOLOGICAL AND COMPUTATIONAL LEARNING DEPARTMENT OF BRAIN AND COGNITIVE SCIENCES

We have simulated the behavior of several arti cial ies, interacting visually with each other. Each y is described by a simple tracking system (Poggio and Reichardt, 1973; Land and Collett, 1974) which summarizes behavioral experiments in which individual ies xate a target. Our main nding is that the interaction of these simple modules gives rise to a variety of relatively complex behaviors. In particular, we observe a swarm-like behavior of a group of many arti cial ies for certain reasonable ranges of our tracking system parameters. Copyright c Massachusetts Institute of Technology, 1994 This report describes research done at the Arti cial Intelligence Laboratory and within the Center for Biological and Computational Learning in the Department of Brain and Cognitive Sciences at the Massachusetts Institute of Technology. This research is sponsored by grants from ONR under contract N00014-93-1-0385 and from ARPA-ONR under contract N00014-92-J-1879; and by a grant from the National Science Foundation under contract ASC-9217041 (this award includes funds from ARPA provided under the HPCC program). Additional support is provided by the North Atlantic Treaty Organization, ATR Audio and Visual Perception Research Laboratories, Mitsubishi Electric Corporation, Sumitomo Metal Industries, and Siemens AG. Support for the A.I. Laboratory's arti cial intelligence research is provided by ARPA-ONR contract N00014-91-J-4038. Tomaso Poggio is supported by the Uncas and Helen Whitaker Chair at MIT's Whitaker College. Martino Poggio is with the Roxbury Latin School, St. Theresa Ave., West Roxbury, MA

[1]  A. Wald.  Contributions to the Theory of Statistical Estimation and Testing Hypotheses , 1939 .

[2]  Dr. M. G. Worster.  Methods of Mathematical Physics , 1947, Nature.

[3]  Le Cam,et al.  On some asymptotic properties of maximum likelihood estimates and related Bayes' estimates , 1953 .

[4]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[5]  O. Schade.  Optical and photoelectric analog of the eye. , 1956, Journal of the Optical Society of America.

[6]  J. Lamperti.  ON CONVERGENCE OF STOCHASTIC PROCESSES , 1962 .

[7]  A Tikhonov,et al.  Solution of Incorrectly Formulated Problems and the Regularization Method , 1963 .

[8]  R. Newton.  Scattering theory of waves and particles , 1966 .

[9]  J. Robson,et al.  Application of fourier analysis to the visibility of gratings , 1968, The Journal of physiology.

[10]  L. Baum,et al.  A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .

[11]  G. Wahba,et al.  A Correspondence Between Bayesian Estimation on Stochastic Processes and Smoothing by Splines , 1970 .

[12]  K. B. Haley,et al.  Optimization Theory with Applications , 1970 .

[13]  N. Goldenfeld.  Lectures On Phase Transitions And The Renormalization Group , 1972 .

[14]  M. Stone.  Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[15]  David M. Allen,et al.  The Relationship Between Variable Selection and Data Agumentation and a Method for Prediction , 1974 .

[16]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[17]  Noam Chomsky,et al.  The Logical Structure of Linguistic Theory , 1975 .

[18]  C. WEHRHAHN,et al.  Real-time delayed tracking in flies , 1976, Nature.

[19]  D. W. Noid.  Studies in Molecular Dynamics , 1976 .

[20]  M. Stone.  An Asymptotic Equivalence of Choice of Model by Cross‐Validation and Akaike's Criterion , 1977 .

[21]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[22]  R. F. Hess,et al.  The threshold contrast sensitivity function in strabismic amblyopia: Evidence for a two type classification , 1977, Vision Research.

[23]  Abraham Lempel,et al.  Compression of individual sequences via variable-rate coding , 1978, IEEE Trans. Inf. Theory.

[24]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[25]  Akira Kodama.  Mechanism of Color Perception , 1979 .

[26]  J. Baker.  Trainable grammars for speech recognition , 1979 .

[27]  L. Schumaker.  Spline Functions: Basic Theory , 1981 .

[28]  A. Cohen,et al.  Finite Mixture Distributions , 1982 .

[29]  J. Gerard Wolff,et al.  Language acquisition, data compression and generalization , 1982 .

[30]  Abraham Kandel,et al.  Fuzzy techniques in pattern recognition , 1982 .

[31]  Geoffrey E. Hinton,et al.  OPTIMAL PERCEPTUAL INFERENCE , 1983 .

[32]  Morris Halle,et al.  On distinctive features and their articulatory implementation , 1983 .

[33]  W. Nelson Francis,et al.  FREQUENCY ANALYSIS OF ENGLISH USAGE: LEXICON AND GRAMMAR , 1983 .

[34]  G. Wahba.  Bayesian "Confidence Intervals" for the Cross-validated Smoothing Spline , 1983 .

[35]  Raymond A. DeCarlo,et al.  Continuation methods: Theory and applications , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[36]  R. Dudley.  A course on empirical processes , 1984 .

[37]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Shun-ichi Amari,et al.  Differential-geometrical methods in statistics , 1985 .

[39]  C Snow,et al.  Child language data exchange system , 1984, Journal of Child Language.

[40]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[41]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[42]  Geoffrey E. Hinton,et al.  Learning and relearning in Boltzmann machines , 1986 .

[43]  L. L. Cam,et al.  Asymptotic Methods In Statistical Decision Theory , 1986 .

[44]  A. Verri,et al.  Regularization Theory and Shape Constraints , 1986 .

[45]  D. Amit,et al.  Statistical mechanics of neural networks near saturation , 1987 .

[46]  Sergei Ovchinnikov,et al.  Fuzzy sets and applications , 1987 .

[47]  Richard Durbin,et al.  An analogue approach to the travelling salesman problem using an elastic net method , 1987, Nature.

[48]  Lawrence Davis,et al.  Genetic Algorithms and Simulated Annealing , 1987 .

[49]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[50]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[51]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[52]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[53]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[54]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[55]  A. Sitenko,et al.  DIRECT NUCLEAR REACTIONS , 1990 .

[56]  Rose,et al.  Statistical mechanics and phase transitions in clustering. , 1990, Physical review letters.

[57]  D. J. Wallace.  Statistical field theory. Volumes 1 and 2 , 1990 .

[58]  G. Wahba.  Spline models for observational data , 1990 .

[59]  Alan L. Yuille,et al.  Generalized Deformable Models, Statistical Physics, and Matching Problems , 1990, Neural Computation.

[60]  Tomaso A. Poggio,et al.  Extensions of a Theory of Networks for Approximation and Learning , 1990, NIPS.

[61]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[62]  S. E. Hills,et al.  Illustration of Bayesian Inference in Normal Data Models Using Gibbs Sampling , 1990 .

[63]  Grace L. Yang,et al.  Asymptotics In Statistics , 1990 .

[64]  Grace Wahba,et al.  Spline Models for Observational Data , 1990 .

[65]  Yaser S. Abu-Mostafa,et al.  Learning from hints in neural networks , 1990, J. Complex..

[66]  R. Bishop,et al.  Quantum many-particle systems , 1990 .

[67]  Michel Le Bellac,et al.  Quantum and statistical field theory , 1991 .

[68]  R. Balian.  From microphysics to macrophysics , 1991 .

[69]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[70]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[71]  Andrew L. Rukhin,et al.  Tools for statistical inference , 1991 .

[72]  Jocelyn Sietsma,et al.  Creating artificial neural networks that generalize , 1991, Neural Networks.

[73]  M. Ferraro.  Invariant Pattern Representations and Lie Groups Theory , 1992 .

[74]  Fernando Pereira,et al.  Inside-Outside Reestimation From Partially Bracketed Corpora , 1992, HLT.

[75]  Pfadintegrale in der Quantenphysik , 1992 .

[76]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[77]  A. Fisher,et al.  The Theory of critical phenomena , 1992 .

[78]  Ronald R. Coifman,et al.  Entropy-based algorithms for best basis selection , 1992, IEEE Trans. Inf. Theory.

[79]  David J. C. MacKay,et al.  The Evidence Framework Applied to Classification Networks , 1992, Neural Computation.

[80]  Glenn Carroll,et al.  Learn-ing probaballstic dependency grammars from labelled text , 1992 .

[81]  W. Härdle.  Applied Nonparametric Regression , 1992 .

[82]  William H. Press,et al.  Numerical recipes in C (2nd ed.): the art of scientific computing , 1992 .

[83]  Yaser S. Abu-Mostafa,et al.  A Method for Learning From Hints , 1992, NIPS.

[84]  David J. C. MacKay,et al.  A Practical Bayesian Framework for Backpropagation Networks , 1992, Neural Computation.

[85]  M. Bernhard.  Introduction to Chaotic Dynamical Systems , 1992 .

[86]  Jeffrey Mark Siskind,et al.  Naive physics, event perception, lexical semantics, and language acquisition , 1992 .

[87]  T. Poggio,et al.  3D Object Recognition: Symmetry and Virtual Views , 1992 .

[88]  David Haussler,et al.  Decision Theoretic Generalizations of the PAC Model for Neural Net and Other Learning Applications , 1992, Inf. Comput..

[89]  C. Beck,et al.  Thermodynamics of chaotic systems , 1993 .

[90]  Roberto Brunelli,et al.  Face Recognition: Features Versus Templates , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[91]  T. Watkin,et al.  THE STATISTICAL-MECHANICS OF LEARNING A RULE , 1993 .

[92]  J. Besag,et al.  Spatial Statistics and Bayesian Computation , 1993 .

[93]  Jeffrey Mark Siskind.  Lexical Acquisition as Constraint Satisfaction , 1993 .

[94]  Morris Halle,et al.  Distributed morphology and the pieces of inflection , 1993 .

[95]  Tomaso Poggio,et al.  Example Based Image Analysis and Synthesis , 1993 .

[96]  David H. Wolpert,et al.  Bayesian Backpropagation Over I-O Functions Rather Than Weights , 1993, NIPS.

[97]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[98]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[99]  Yaser S. Abu-Mostafa,et al.  Hints and the VC Dimension , 1993, Neural Computation.

[100]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.

[101]  Jeffrey Mark Siskind,et al.  Lexical Acquisition in the Presence of Noise and Homonymy , 1994, AAAI.

[102]  Andrew R. Webb,et al.  Functional approximation by feed-forward networks: a least-squares approach to generalization , 1994, IEEE Trans. Neural Networks.

[103]  Alan L. Yuille,et al.  Statistical Physics, Mixtures of Distributions, and the EM Algorithm , 1994, Neural Computation.

[104]  Martin Kay,et al.  Regular Models of Phonological Rule Systems , 1994, CL.

[105]  C. Snow,et al.  Input and interaction in language acquisition: The changing role of negative evidence in theories of language development , 1994 .

[106]  Carl de Marcken.  The Acquisition of a Lexicon from Paired Phoneme Sequences and Semantic Representations , 1994, ICGI.

[107]  A. Cutler.  Segmentation problems, rhythmic solutions * , 1994 .

[108]  Umesh V. Vazirani,et al.  An Introduction to Computational Learning Theory , 1994 .

[109]  David Beymer,et al.  Face recognition under varying pose , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[110]  B. Silverman,et al.  Nonparametric regression and generalized linear models , 1994 .

[111]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .

[112]  Michael Kenstowicz,et al.  Phonology In Generative Grammar , 1994 .

[113]  J. J. Kosowsky,et al.  Statistical Physics Algorithms That Converge , 1994, Neural Computation.

[114]  Michael C. Burl,et al.  Finding faces in cluttered scenes using random labeled graph matching , 1995, Proceedings of IEEE International Conference on Computer Vision.

[115]  Carl E. Rasmussen,et al.  In Advances in Neural Information Processing Systems , 2011 .

[116]  Peter L. Balise,et al.  Boolean Algebra and Its Applications , 1995 .

[117]  Frédéric Bimbot,et al.  Language modeling by variable length sequences: theoretical formulation and evaluation of multigrams , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[118]  Rudolf Podgornik,et al.  Statistical thermodynamics of surfaces, interfaces, and membranes , 1995 .

[119]  Todd K. Leen,et al.  From Data Distributions to Regularization in Invariant Learning , 1995, Neural Computation.

[120]  Geoffrey E. Hinton,et al.  The Helmholtz Machine , 1995, Neural Computation.

[121]  Geoffrey E. Hinton,et al.  The "wake-sleep" algorithm for unsupervised neural networks. , 1995, Science.

[122]  Eric Sven Ristad,et al.  New Techniques for Context Modeling , 1995, ACL.

[123]  Inhomogeneous Random Phase Approximation: A Solvable Model , 1995 .

[124]  F. Girosi,et al.  Prior knowledge and the creation of "virtual" examples for RBF networks , 1995, Proceedings of 1995 IEEE Workshop on Neural Networks for Signal Processing.

[125]  Shun-ichi Amari,et al.  Information geometry of the EM and em algorithms for neural networks , 1995, Neural Networks.

[126]  Yoshua Bengio,et al.  Pattern Recognition and Neural Networks , 1995 .

[127]  Hill,et al.  Annealed Theories of Learning , 1995 .

[128]  William T. Freeman,et al.  Presented at: 2nd Annual IEEE International Conference on Image , 1995 .

[129]  W. Ebeling.  Stochastic Processes in Physics and Chemistry , 1995 .

[130]  Hans-Paul Schwefel,et al.  Evolution and optimum seeking , 1995, Sixth-generation computer technology series.

[131]  Valeriu Beiu,et al.  Density Estimation as a Preprocessing Step for Constructive Algorithms , 1995, SNN Symposium on Neural Networks.

[132]  Physics of critical fluctuations , 1995 .

[133]  Inhomogeneous Random Phase Approximation for Nuclear and Atomic Reactions , 1995 .

[134]  Laurenz Wiskott,et al.  Labeled graphs and dynamic link matching for face recognition and scene analysis , 1995 .

[135]  Michael I. Jordan.  Why the logistic function? A tutorial discussion on probabilities and neural networks , 1995 .

[136]  Christopher M. Bishop,et al.  Current address: Microsoft Research, , 2022 .

[137]  Carl de Marcken,et al.  Lexical Heads, Phrase Structure and the Induction of Grammar , 1995, VLC@ACL.

[138]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[139]  Tomaso A. Poggio,et al.  Regularization Theory and Neural Networks Architectures , 1995, Neural Computation.

[140]  Stanley F. Chen,et al.  Bayesian Grammar Induction for Language Modeling , 1995, ACL.

[141]  Michael I. Jordan,et al.  Mean Field Theory for Sigmoid Belief Networks , 1996, J. Artif. Intell. Res..

[142]  David H. Wolpert,et al.  The Existence of A Priori Distinctions Between Learning Algorithms , 1996, Neural Computation.

[143]  Alexander J. Smola,et al.  Support Vector Method for Function Approximation, Regression Estimation and Signal Processing , 1996, NIPS.

[144]  Dimitar P. Filev,et al.  Fuzzy SETS AND FUZZY LOGIC , 1996 .

[145]  San Cristóbal Mateo,et al.  The Lack of A Priori Distinctions Between Learning Algorithms , 1996 .

[146]  Richard M. Golden,et al.  Mathematical Methods for Neural Network Analysis and Design , 1996 .

[147]  John Shawe-Taylor,et al.  A framework for structural risk minimisation , 1996, COLT '96.

[148]  Penio S. Penev,et al.  Local feature analysis: A general statistical theory for object representation , 1996 .

[149]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[150]  Callan,et al.  Field Theories for Learning Probability Distributions. , 1996, Physical review letters.

[151]  David Barber,et al.  Gaussian Processes for Bayesian Classification via Hybrid Monte Carlo , 1996, NIPS.

[152]  V. Buzek,et al.  From quantum Bayesian inference to quantum tomography , 1997, quant-ph/9701029.

[153]  Michael I. Jordan,et al.  Computing upper and lower bounds on likelihoods in intractable networks , 1996, UAI.

[154]  R W Prager,et al.  Development of low entropy coding in a recurrent network. , 1996, Network.

[155]  A. Murat Tekalp,et al.  Face detection and facial feature extraction using color, shape and symmetry-based cost functions , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[156]  Eytan Domany,et al.  Data Clustering Using a Model Granular Magnet , 1997, Neural Computation.

[157]  Joachim M. Buhmann,et al.  Pairwise Data Clustering by Deterministic Annealing , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[158]  Federico Girosi,et al.  Support Vector Machines: Training and Applications , 1997 .

[159]  Dorothea Heiss-Czedik,et al.  An Introduction to Genetic Algorithms. , 1997, Artificial Life.

[160]  Radford M. Neal.  Monte Carlo Implementation of Gaussian Process Models for Bayesian Regression and Classification , 1997, physics/9701026.

[161]  Vijay Balasubramanian,et al.  Statistical Inference, Occam's Razor, and Statistical Mechanics on the Space of Probability Distributions , 1996, Neural Computation.

[162]  John D. Lafferty,et al.  Inducing Features of Random Fields , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[163]  Tomaso A. Poggio,et al.  Pedestrian detection using wavelet templates , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[164]  Alex Pentland,et al.  Parametrized structure from motion for 3D adaptive feedback tracking of faces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[165]  C. Papageorgiou,et al.  Object and pattern detection in video sequences , 1997 .

[166]  Vladimir Cherkassky,et al.  The Nature Of Statistical Learning Theory , 1997, IEEE Trans. Neural Networks.

[167]  Terrence J. Sejnowski,et al.  Learning Nonlinear Overcomplete Representations for Efficient Coding , 1997, NIPS.

[168]  Bernhard Schölkopf,et al.  On a Kernel-Based Method for Pattern Recognition, Regression, Approximation, and Operator Inversion , 1998, Algorithmica.

[169]  Tomaso A. Poggio,et al.  Example-Based Learning for View-Based Human Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[170]  Tomaso A. Poggio,et al.  A Sparse Representation for Function Approximation , 1998, Neural Computation.

[171]  John Shawe-Taylor,et al.  Structural Risk Minimization Over Data-Dependent Hierarchies , 1998, IEEE Trans. Inf. Theory.

[172]  Takeo Kanade,et al.  Rotation invariant neural network-based face detection , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[173]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[174]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[175]  Takio Kurita,et al.  Scale and Rotation Invariant Recognition Method Using Higher-Order Local Autocorrelation Features of Log-Polar Image , 1998, ACCV.

[176]  J. C. BurgesChristopher.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998 .

[177]  Jorma Rissanen,et al.  Stochastic Complexity in Statistical Inquiry , 1989, World Scientific Series in Computer Science.

[178]  Tomaso A. Poggio,et al.  A general framework for object detection , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[179]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[180]  Takeo Kanade,et al.  Probabilistic modeling of local appearance and spatial relationships for object recognition , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[181]  Federico Girosi,et al.  An Equivalence Between Sparse Approximation and Support Vector Machines , 1998, Neural Computation.

[182]  Tomaso Poggio,et al.  Incorporating prior information in machine learning by creating virtual examples , 1998, Proc. IEEE.

[183]  Kurt Binder,et al.  Monte Carlo Simulation in Statistical Physics , 1992, Graduate Texts in Physics.

[184]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .