Elements of artificial neural networks

History of neural networks supervised learning - single-layer networks supervised learning - multilayer networks I supervised learning - multilayer networks II unsupervised learning associative models optimization methods a little math data.

[1]  W. Pitts,et al.  A Statistical Consequence of the Logical Calculus of Nervous Nets , 1943 .

[2]  Dennis Gabor,et al.  Theory of communication , 1946 .

[3]  R. Courant,et al.  Methods of Mathematical Physics , 1962 .

[4]  Norbert Wiener,et al.  Cybernetics: Control and Communication in the Animal and the Machine. , 1949 .

[5]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1951 .

[6]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[7]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[8]  B. Cragg,et al.  The organisation of neurones: a co-operative analogy. , 1954, Electroencephalography and clinical neurophysiology.

[9]  Dennis Gabor,et al.  Communication Theory and Cybernetics , 1954 .

[10]  J. von Neumann,et al.  Probabilistic Logic and the Synthesis of Reliable Organisms from Unreliable Components , 1956 .

[11]  A. M. Uttley,et al.  Conditional Probability Machines and Conditioned Reflexes , 1956 .

[12]  A. M. Uttley,et al.  Temporal and Spatial Patterns in a Conditional Probability Machine , 1956 .

[13]  R. L. Beurle.  Properties of a mass of cells capable of regenerating pulses , 1956, Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences.

[14]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[15]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[16]  S. Dreyfus.  The numerical solution of variational problems , 1962 .

[17]  Frank Rosenblatt,et al.  PRINCIPLES OF NEURODYNAMICS. PERCEPTRONS AND THE THEORY OF BRAIN MECHANISMS , 1963 .

[18]  W. Rudin.  Principles of mathematical analysis , 1964 .

[19]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[20]  W. K. Taylor.  Cortico-thalamic organization and memory , 1964, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[21]  Thomas M. Cover,et al.  Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition , 1965, IEEE Trans. Electron. Comput..

[22]  A M Uttley,et al.  The transmission of information and the effect of local feedback in theoretical and neural networks. , 1966, Brain research.

[23]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[24]  Shun-ichi Amari,et al.  A Theory of Adaptive Pattern Classifiers , 1967, IEEE Trans. Electron. Comput..

[25]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[26]  J. D. Cowan,et al.  Statistical Mechanics of Nervous Nets , 1968 .

[27]  William G. Wee,et al.  Generalized Inverse Approach to Adaptive Multiclass Pattern Classification , 1968, IEEE Transactions on Computers.

[28]  Arthur E. Bryson,et al.  Applied Optimal Control , 1969 .

[29]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[30]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[31]  J. Neter,et al.  Applied linear statistical models : regression, analysis of variance, and experimental designs , 1974 .

[32]  W. Vent,et al.  Rechenberg, Ingo, Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. 170 S. mit 36 Abb. Frommann‐Holzboog‐Verlag. Stuttgart 1973. Broschiert , 1975 .

[33]  B. Widrow,et al.  Adaptive noise cancelling: Principles and applications , 1975 .

[34]  C. Malsburg,et al.  How patterned neural connections can be set up by self-organization , 1976, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[35]  Stephen A. Ritz,et al.  Distinctive features, categorical perception, and probability learning: some applications of a neural model , 1977 .

[36]  D. Hubel,et al.  Ferrier lecture - Functional architecture of macaque monkey visual cortex , 1977, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[37]  S. Amari,et al.  A Mathematical Foundation for Statistical Neurodynamics , 1977 .

[38]  A. G. Vitushkin.  On representation of functions by means of superpositions and related topics , 1978 .

[39]  Roman Bek,et al.  Discourse on one way in which a quantum-mechanics language on the classical logical base can be built up , 1978, Kybernetika.

[40]  Shun-Ichi Amari,et al.  Topographic organization of nerve fields , 1979, Neuroscience Letters.

[41]  Robert M. Gray,et al.  Locally Optimal Block Quantizer Design , 1980, Inf. Control..

[42]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[43]  S. Grossberg.  Studies of mind and brain : neural principles of learning, perception, development, cognition, and motor control , 1982 .

[44]  Stephen Grossberg,et al.  Classical and Instrumental Learning by Neural Networks , 1982 .

[45]  Stephen Grossberg,et al.  Absolute stability of global pattern formation and parallel memory storage by competitive neural networks , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[46]  Takayuki Ito,et al.  Neocognitron: A neural network model for a mechanism of visual pattern recognition , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[47]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[48]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[49]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Shokri Z. Selim,et al.  K-Means-Type Algorithms: A Generalized Convergence Theorem and Characterization of Local Optimality , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  E. Oja,et al.  On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix , 1985 .

[52]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[53]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[54]  Geoffrey E. Hinton,et al.  A general framework for parallel distributed processing , 1986 .

[55]  Richard P. Lippmann,et al.  An introduction to computing with neural nets , 1987 .

[56]  B. Kosco.  Differential Hebbian learning , 1987 .

[57]  Colin Giles,et al.  Learning, invariance, and generalization in high-order neural networks. , 1987, Applied optics.

[58]  Helge Ritter,et al.  Topology conserving mappings for learning motor tasks , 1987 .

[59]  Carsten Peterson,et al.  A Mean Field Theory Learning Algorithm for Neural Networks , 1987, Complex Syst..

[60]  K. Fukushima.  Neural network model for selective attention in visual pattern recognition and associative recall. , 1987, Applied optics.

[61]  Richard Lippmann,et al.  Neural Net and Traditional Classifiers , 1987, NIPS.

[62]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[63]  A. Guez,et al.  Neural network architecture for control , 1988, IEEE Control Systems Magazine.

[64]  G. Josin,et al.  Robot control using neural networks , 1988, IEEE 1988 International Conference on Neural Networks.

[65]  D. O. Hebb,et al.  The organization of behavior , 1988 .

[66]  Bernard Widrow,et al.  Adaptive switching circuits , 1988 .

[67]  B. Widrow,et al.  An adaptive 'broom balancer' with visual inputs , 1988, IEEE 1988 International Conference on Neural Networks.

[68]  A. R. Spitzer,et al.  Neural network identification and extraction of repetitive superimposed pulses in noisy 1-D signals , 1988 .

[69]  R.J.F. Dow,et al.  Neural net pruning-why and how , 1988, IEEE 1988 International Conference on Neural Networks.

[70]  H. White,et al.  There exists a neural network that does not make avoidable mistakes , 1988, IEEE 1988 International Conference on Neural Networks.

[71]  BART KOSKO,et al.  Bidirectional associative memories , 1988, IEEE Trans. Syst. Man Cybern..

[72]  Scott E. Fahlman,et al.  An empirical study of learning speed in back-propagation networks , 1988 .

[73]  Yann LeCun,et al.  A theoretical framework for back-propagation , 1988 .

[74]  Duane DeSieno,et al.  Adding a conscience to competitive learning , 1988, IEEE 1988 International Conference on Neural Networks.

[75]  Jerome A. Feldman,et al.  Connectionist Models and Their Properties , 1982, Cogn. Sci..

[76]  Stephen Grossberg,et al.  The ART of adaptive pattern recognition by a self-organizing neural network , 1988, Computer.

[77]  James L. McClelland.  Explorations In Parallel Distributed Processing , 1988 .

[78]  Robert Hecht-Nielsen,et al.  Applications of counterpropagation networks , 1988, Neural Networks.

[79]  David Haussler,et al.  What Size Net Gives Valid Generalization? , 1989, Neural Computation.

[80]  G. C. Tiao,et al.  Model Specification in Multivariate Time Series , 1989 .

[81]  David E. Goldberg,et al.  Sizing Populations for Serial and Parallel Genetic Algorithms , 1989, ICGA.

[82]  Emile H. L. Aarts,et al.  Combinatorial Optimization on a Boltzmann Machine , 1989, J. Parallel Distributed Comput..

[83]  Tomaso A. Poggio,et al.  Representation Properties of Networks: Kolmogorov's Theorem Is Irrelevant , 1989, Neural Computation.

[84]  S. Abe.  Theories on the Hopfield neural networks , 1989, International 1989 Joint Conference on Neural Networks.

[85]  Terence D. Sanger,et al.  Optimal unsupervised learning in a single-layer linear feedforward neural network , 1989, Neural Networks.

[86]  Hecht-Nielsen.  Theory of the backpropagation neural network , 1989 .

[87]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[88]  David E. Rumelhart,et al.  Product Units: A Computationally Powerful and Biologically Plausible Extension to Backpropagation Networks , 1989, Neural Computation.

[89]  Geoffrey E. Hinton.  Connectionist Learning Procedures , 1989, Artif. Intell..

[90]  J. Nadal,et al.  Learning in feedforward layered networks: the tiling algorithm , 1989 .

[91]  Jack D. Cowan,et al.  Neural Networks: The Early Days , 1989, NIPS.

[92]  Daniel J. Amit,et al.  Modeling brain function: the world of attractor neural networks, 1st Edition , 1989 .

[93]  Yoh-Han Pao,et al.  Adaptive pattern recognition and neural networks , 1989 .

[94]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[95]  S. P. Luttrell,et al.  Self-organisation: a derivation from first principles of a class of learning algorithms , 1989, International 1989 Joint Conference on Neural Networks.

[96]  Yann LeCun,et al.  Optimal Brain Damage , 1989, NIPS.

[97]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[98]  Patrick K. Simpson,et al.  Artificial Neural Systems: Foundations, Paradigms, Applications, and Implementations , 1990 .

[99]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[100]  Kishan G. Mehrotra,et al.  Sunspot numbers forecasting using neural networks , 1990, Proceedings. 5th IEEE International Symposium on Intelligent Control 1990.

[101]  David E. Rumelhart,et al.  Predicting the Future: a Connectionist Approach , 1990, Int. J. Neural Syst..

[102]  M. Golea,et al.  A Convergence Theorem for Sequential Learning in Two-Layer Perceptrons , 1990 .

[103]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[104]  Mahesan Niranjan,et al.  A theoretical investigation into the performance of the Hopfield model , 1990, IEEE Trans. Neural Networks.

[105]  Rose,et al.  Statistical mechanics and phase transitions in clustering. , 1990, Physical review letters.

[106]  T Poggio,et al.  Regularization Algorithms for Learning That Are Equivalent to Multilayer Networks , 1990, Science.

[107]  Larry J. Eshelman,et al.  The CHC Adaptive Search Algorithm: How to Have Safe Search When Engaging in Nontraditional Genetic Recombination , 1990, FOGA.

[108]  James A. Anderson,et al.  Radar signal categorization using a neural network , 1990, Proc. IEEE.

[109]  Eric Jon Hartman,et al.  A large storage capacity neural network content-addressable memory , 1990 .

[110]  Shun-ichi Amari,et al.  Mathematical foundations of neurocomputing , 1990, Proc. IEEE.

[111]  Jean-Pierre Nadal,et al.  Neural trees: a new tool for classification , 1990 .

[112]  Marcus Frean,et al.  The Upstart Algorithm: A Method for Constructing and Training Feedforward Neural Networks , 1990, Neural Computation.

[113]  Derrick H. Nguyen,et al.  Neural networks for self-learning control systems , 1990 .

[114]  Gerhard W. Dueck,et al.  Threshold accepting: a general purpose optimization algorithm appearing superior to simulated anneal , 1990 .

[115]  John C. Platt,et al.  Networks for the Separation of Sources that Are Superimposed and Delayed , 1991, NIPS.

[116]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[117]  John E. Moody,et al.  The Effective Number of Parameters: An Analysis of Generalization and Regularization in Nonlinear Learning Systems , 1991, NIPS.

[118]  Kishan G. Mehrotra,et al.  Bounds on the number of samples needed for neural learning , 1991, IEEE Trans. Neural Networks.

[119]  Andrew R. Barron,et al.  Complexity Regularization with Application to Artificial Neural Networks , 1991 .

[120]  C. Mohan,et al.  An optimization network for solving a set of simultaneous linear equations , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[121]  S. Oh,et al.  Regularization using jittered training data , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[122]  Vera Kurková,et al.  Kolmogorov's theorem and multilayer neural networks , 1992, Neural Networks.

[123]  José Carlos Príncipe,et al.  The gamma model--A new neural model for temporal processing , 1992, Neural Networks.

[124]  Babak Hassibi,et al.  Second Order Derivatives for Network Pruning: Optimal Brain Surgeon , 1992, NIPS.

[125]  J. D. Schaffer,et al.  Combinations of genetic algorithms and neural networks: a survey of the state of the art , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[126]  E. Oja,et al.  Clustering Properties of Hierarchical Self-Organizing Maps , 1992 .

[127]  Kishan G. Mehrotra,et al.  Forecasting the behavior of multivariate time series using neural networks , 1992, Neural Networks.

[128]  S. P. Luttrell,et al.  Self-supervised adaptive networks , 1992 .

[129]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[130]  Thomas Martinetz,et al.  'Neural-gas' network for vector quantization and its application to time-series prediction , 1993, IEEE Trans. Neural Networks.

[131]  Lester Ingber,et al.  Simulated annealing: Practice versus theory , 1993 .

[132]  Andrew H. Gee,et al.  An analytical framework for optimizing neural networks , 1993, Neural Networks.

[133]  Chilukuri K. Mohan,et al.  Multiprocessor Document Allocation: a Neural Network Approach , 1994 .

[134]  Juha Karhunen,et al.  Representation and separation of signals using nonlinear PCA type learning , 1994, Neural Networks.

[135]  R. Palmer,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[136]  J.G. Delgado-Frias,et al.  Impact of energy function on a neural network model for optimization problems , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[137]  J. Shewchuk.  An Introduction to the Conjugate Gradient Method Without the Agonizing Pain , 1994 .

[138]  Kagan Tumer,et al.  Structural adaptation and generalization in supervised feed-forward networks , 1994 .

[139]  Bernd Fritzke,et al.  Growing cell structures--A self-organizing network for unsupervised and supervised learning , 1994, Neural Networks.

[140]  Joydeep Ghosh,et al.  Ridge polynomial networks , 1995, IEEE Trans. Neural Networks.

[141]  Amnon Barak,et al.  Selectively Destructive Re-start , 1995, International Conference on Genetic Algorithms.

[142]  Bart Kosko,et al.  Neural networks and fuzzy systems , 1998 .

[143]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .