Artificial Neural Networks

Artificial neural networks (ANNs) constitute a class of flexible nonlinear models designed to mimic biological neural systems. In this entry, we introduce ANN using familiar econometric terminology and provide an overview of ANN modeling approach and its implementation methods. † Correspondence: Chung-Ming Kuan, Institute of Economics, Academia Sinica, 128 Academia Road, Sec. 2, Taipei 115, Taiwan; ckuan@econ.sinica.edu.tw. †† I would like to express my sincere gratitude to the editor, Professor Steven Durlauf, for his patience and constructive comments on early drafts of this entry. I also thank Shih-Hsun Hsu and Yu-Lieh Huang for very helpful suggestions. The remaining errors are all mine.

[1]  Carla H. Lagorio,et al.  Psychology , 1929, Nature.

[2]  Richard Southwell The Imperial College , 1949 .

[3]  M. Rosenblatt Remarks on Some Nonparametric Estimates of a Density Function , 1956 .

[4]  J. Kiefer Optimum Sequential Search and Approximation Methods Under Minimum Regularity Assumptions , 1957 .

[5]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[6]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[7]  Roger Fletcher,et al.  A Rapidly Convergent Descent Method for Minimization , 1963, Comput. J..

[8]  M. J. D. Powell,et al.  An efficient method for finding the minimum of a function of several variables without calculating derivatives , 1964, Comput. J..

[9]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[10]  M. Aizerman,et al.  Theoretical Foundations of the Potential Function Method in Pattern Recognition Learning , 1964 .

[11]  A. Noll Short‐Time Spectrum and “Cepstrum” Techniques for Vocal‐Pitch Detection , 1964 .

[12]  J. V. Dennis WOODPECKER DAMAGE TO UTILITY POLES: SPECIAL REFERENCE TO THE ROLE OF TERRITORY AND RESONANCE WITH , 1964 .

[13]  M. Sondhi,et al.  New methods of pitch extraction , 1968 .

[14]  James Joseph Biundo,et al.  Analysis of Contingency Tables , 1969 .

[15]  R. Zahradnik THEORY AND TECHNIQUES OF OPTIMIZATION FOR PRACTICING ENGINEERS. , 1971 .

[16]  A. G. Ivakhnenko,et al.  Polynomial Theory of Complex Systems , 1971, IEEE Trans. Syst. Man Cybern..

[17]  Donald G. Childers,et al.  Signal detection and extraction by cepstrum techniques , 1972, IEEE Trans. Inf. Theory.

[18]  Nils J. Nilsson,et al.  Artificial Intelligence , 1974, IFIP Congress.

[19]  M. Ross,et al.  Average magnitude difference function pitch extractor , 1974 .

[20]  James P. Egan,et al.  Signal detection theory and ROC analysis , 1975 .

[21]  M. J. Cheng,et al.  Comparative performance study of several pitch detection algorithms , 1975 .

[22]  Ronald W. Schafer,et al.  Real-time digital hardware pitch detector , 1976 .

[23]  B. R. Hergenhahn,et al.  An Introduction to Theories of Learning , 2020 .

[24]  Lawrence R. Rabiner,et al.  On the use of autocorrelation analysis for pitch detection , 1977 .

[25]  S. Boll,et al.  Suppression of acoustic noise in speech using spectral subtraction , 1979 .

[26]  Ramakant Khazanie,et al.  Elementary statistics, in a world of applications , 1979 .

[27]  Jeremy I. Bulow Durable-Goods Monopolists , 1982, Journal of Political Economy.

[28]  Josef Kittler,et al.  Pattern recognition : a statistical approach , 1982 .

[29]  Wolfgang Hess,et al.  Pitch Determination of Speech Signals , 1983 .

[30]  M. F.,et al.  Bibliography , 1985, Experimental Gerontology.

[31]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[32]  J. Rissanen Stochastic Complexity and Modeling , 1986 .

[33]  S. Grossberg,et al.  Neural Dynamics of Category Learning and Recognition: Attention, Memory Consolidation, and Amnesia , 1987 .

[34]  H. White,et al.  A Unified Theory of Estimation and Inference for Nonlinear Dynamic Models , 1988 .

[35]  Robert A. Jacobs,et al.  Increased rates of convergence through learning rate adaptation , 1987, Neural Networks.

[36]  Alberto L. Sangiovanni-Vincentelli,et al.  Efficient Parallel Learning Algorithms for Neural Networks , 1988, NIPS.

[37]  H. White Some Asymptotic Results for Learning in Single Hidden-Layer Feedforward Network Models , 1989 .

[38]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[39]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[40]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[41]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[42]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[43]  Patrick K. Simpson,et al.  Neural networks for sonar signal processing , 1990 .

[44]  Kurt Hornik,et al.  Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks , 1990, Neural Networks.

[45]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[46]  A. Agresti An introduction to categorical data analysis , 1997 .

[47]  William C. Davidon,et al.  Variable Metric Method for Minimization , 1959, SIAM J. Optim..

[48]  Kurt Hornik,et al.  Convergence of learning algorithms with constant learning rates , 1991, IEEE Trans. Neural Networks.

[49]  Leslie Greengard,et al.  The Fast Gauss Transform , 1991, SIAM J. Sci. Comput..

[50]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[51]  Myung Won Kim,et al.  The effect of initial weights on premature saturation in back-propagation learning , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[52]  Andrew R. Barron,et al.  Complexity Regularization with Application to Artificial Neural Networks , 1991 .

[53]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[54]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[55]  David W. Scott,et al.  Feasibility of multivariate density estimates , 1991 .

[56]  D. W. Scott,et al.  Multivariate Density Estimation, Theory, Practice and Visualization , 1992 .

[57]  John G. Proakis,et al.  Digital Signal Processing: Principles, Algorithms, and Applications , 1992 .

[58]  Chung-Ming Kuan,et al.  Forecasting exchange rates using feedforward and recurrent neural networks , 1992 .

[59]  M.A. Sid-Ahmed,et al.  LBAQ: a pattern recognition neural network that learns by asking questions , 1992, [1992] Proceedings of the 35th Midwest Symposium on Circuits and Systems.

[60]  D Mumford,et al.  On the computational architecture of the neocortex. II. The role of cortico-cortical loops. , 1992, Biological cybernetics.

[61]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[62]  Timothy Masters,et al.  Practical neural network recipes in C , 1993 .

[63]  G. Hoogenboom The Georgia Automated Environmental Monitoring Network , 1993 .

[64]  Kurt Hornik,et al.  Some new results on neural network approximation , 1993, Neural Networks.

[65]  Joab R Winkler,et al.  Numerical recipes in C: The art of scientific computing, second edition , 1993 .

[66]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[67]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[68]  R. E. Abdel-Aal,et al.  A machine-learning approach to modelling and forecasting the minimum temperature at Dhahran, Saudi Arabia , 1994 .

[69]  Halbert White,et al.  Artificial neural networks: an econometric perspective ∗ , 1994 .

[70]  G. Kane Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol 1: Foundations, vol 2: Psychological and Biological Models , 1994 .

[71]  Kurt Hornik,et al.  A Convergence Result for Learning in Recurrent Neural Networks , 1994, Neural Computation.

[72]  D. Signorini,et al.  Neural networks , 1995, The Lancet.

[73]  Chung-Ming Kuan A recurrent Newton algorithm and its convergence properties , 1995, IEEE Trans. Neural Networks.

[74]  Patricia S. O Sullivan,et al.  100 Statistical Tests , 1995 .

[75]  Yasuaki Nakano,et al.  Head and stem extraction from printed music scores using a neural network approach , 1995, Proceedings of 3rd International Conference on Document Analysis and Recognition.

[76]  M. C. Jones,et al.  A Brief Survey of Bandwidth Selection for Density Estimation , 1996 .

[77]  Forecasting earth surface temperature for the optimal application of frost protection methods , 1996 .

[78]  J. Mark Introduction to radial basis function networks , 1996 .

[79]  Anil K. Jain,et al.  Artificial Neural Networks: A Tutorial , 1996, Computer.

[80]  Thomas G. Dietterich What is machine learning? , 2020, Archives of Disease in Childhood.

[81]  Raúl Rojas,et al.  Neural Networks - A Systematic Introduction , 1996 .

[82]  Stephen Chiu,et al.  A Comparative Review of Bandwidth Selection for Kernel Density Estimation , 1996 .

[83]  Anil K. Jain,et al.  Feature Selection: Evaluation, Application, and Small Sample Performance , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[84]  Norman R. Swanson,et al.  A Model Selection Approach to Real-Time Macroeconomic Forecasting Using Linear Models and Artificial Neural Networks , 1997, Review of Economics and Statistics.

[85]  Chun-Chieh Yang,et al.  APPLICATION OF ARTIFICIAL NEURAL NETWORKS FOR SIMULATION OF SOIL TEMPERATURE , 1997 .

[86]  N. Mort,et al.  A neural network system for the protection of citrus crops from frost damage , 1997 .

[87]  An analytical model for the prediction of nocturnal and dawn surface temperatures under calm, clear sky conditions , 1997 .

[88]  Bird hazard detection with airport surveillance radar , 1997 .

[89]  Huien Han,et al.  Estimation of daily soil water evaporation using an artificial neural network , 1997 .

[90]  Terry L King A Guide to Chi-Squared Testing , 1997 .

[91]  H. C. Card,et al.  Birdsong recognition using backpropagation and multivariate statistics , 1997, IEEE Trans. Signal Process..

[92]  Michael I. Jordan Serial Order: A Parallel Distributed Processing Approach , 1997 .

[93]  Vincent Kanade,et al.  Clustering Algorithms , 2021, Wireless RF Energy Transfer in the Massive IoT Era.

[94]  Principles of Freeze Protection for Fruit Crops , 2022 .