Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability

Within this text neural networks are considered as massively interconnected nonlinear adaptive filters.

[1]  A. D. Hestenes The extrapolation, interpolation and smoothing of stationary time series with engineering applications: by Norbert Wiener. 163 pages, 15 × 24 cm. New York, John Wiley & Sons, Inc., 1949. Price, $4.00 , 1950 .

[2]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[3]  E. Lorenz Deterministic nonperiodic flow , 1963 .

[4]  J. H. Wilkinson The algebraic eigenvalue problem , 1966 .

[5]  D. Sprecher On the structure of continuous functions of several variables , 1965 .

[6]  E. M. Wright,et al.  Adaptive Control Processes: A Guided Tour , 1961, The Mathematical Gazette.

[7]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .

[8]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[9]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[10]  J. Makhoul,et al.  Linear prediction: A tutorial review , 1975, Proceedings of the IEEE.

[11]  T. Apostol Modular Functions and Dirichlet Series in Number Theory , 1976 .

[12]  E. Jury Stability of multidimensional scalar and matrix polynomials , 1978, Proceedings of the IEEE.

[13]  Stephen A. Billings,et al.  Identi cation of nonlinear systems-A survey , 1980 .

[14]  R. Mañé,et al.  On the dimension of the compact invariant sets of certain non-linear maps , 1981 .

[15]  Philip E. Gill,et al.  Practical optimization , 1981 .

[16]  M. Schetzen,et al.  Nonlinear system modeling based on the Wiener theory , 1981, Proceedings of the IEEE.

[17]  Stephen Grossberg,et al.  Classical and Instrumental Learning by Neural Networks , 1982 .

[18]  Lennart Ljung,et al.  Theory and Practice of Recursive Identification , 1983 .

[19]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[20]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[21]  Lennart Ljung Analysis of stochastic gradient algorithms for linear regression problems , 1984, IEEE Trans. Inf. Theory.

[22]  Harold J. Kushner,et al.  Approximation and Weak Convergence Methods for Random Processes , 1984 .

[23]  R. Nitzberg Application of the Normalized LMS Algorithm to MSLC , 1985, IEEE Transactions on Aerospace and Electronic Systems.

[24]  Floris Takens,et al.  On the numerical determination of the dimension of an attractor , 1985 .

[25]  I. J. Leontaritis,et al.  Input-output parametric models for non-linear systems Part II: stochastic non-linear systems , 1985 .

[26]  J. P. Lasalle The stability and control of discrete processes , 1986 .

[27]  Neil J. Bershad,et al.  Analysis of the normalized LMS algorithm with Gaussian inputs , 1986, IEEE Trans. Acoust. Speech Signal Process..

[28]  R. Devaney An Introduction to Chaotic Dynamical Systems , 1990 .

[29]  Henry Stark,et al.  Probability, Random Processes, and Estimation Theory for Engineers , 1995 .

[30]  R. Hecht-Nielsen Kolmogorov''s Mapping Neural Network Existence Theorem , 1987 .

[31]  H. Szu,et al.  Nonconvex optimization by fast simulated annealing , 1987, Proceedings of the IEEE.

[32]  P. Vaidyanathan,et al.  A unified structural interpretation of some well-known stability-test procedures for linear systems , 1987, Proceedings of the IEEE.

[33]  R. Lippmann,et al.  An introduction to computing with neural nets , 1987, IEEE ASSP Magazine.

[34]  C. Johnson,et al.  Theory and design of adaptive filters , 1987 .

[35]  Fernando J. Pineda,et al.  GENERALIZATION OF BACKPROPAGATION TO RECURRENT AND HIGH-ORDER NETWORKS. , 1987 .

[36]  Bernard Widrow,et al.  Adaptive switching circuits , 1988 .

[37]  M. B. Priestley,et al.  Non-linear and non-stationary time series analysis , 1990 .

[38]  E. Zeidler Nonlinear functional analysis and its applications , 1988 .

[39]  Robert A. Jacobs,et al.  Increased rates of convergence through learning rate adaptation , 1987, Neural Networks.

[40]  Arie Feuer,et al.  Convergence and performance analysis of the normalized LMS algorithm with uncorrelated Gaussian data , 1988, IEEE Trans. Inf. Theory.

[41]  J. Shynk,et al.  Analysis of the data-reusing LMS algorithm , 1989, Proceedings of the 32nd Midwest Symposium on Circuits and Systems,.

[42]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[43]  S. Sastry,et al.  Adaptive Control: Stability, Convergence and Robustness , 1989 .

[44]  Ronald J. Williams,et al.  Experimental Analysis of the Real-time Recurrent Learning Algorithm , 1989 .

[45]  Tomaso A. Poggio,et al.  Representation Properties of Networks: Kolmogorov's Theorem Is Irrelevant , 1989, Neural Computation.

[46]  Yu He,et al.  Asymptotic Convergence of Backpropagation: Numerical Experiments , 1989, NIPS.

[47]  V.Z. Marmarelis,et al.  Signal transformation and coding in neural systems , 1989, IEEE Transactions on Biomedical Engineering.

[48]  Fernando J. Pineda,et al.  Recurrent Backpropagation and the Dynamical Approach to Adaptive Neural Computation , 1989, Neural Computation.

[49]  Geoffrey E. Hinton,et al.  Phoneme recognition using time-delay neural networks , 1989, IEEE Trans. Acoust. Speech Signal Process..

[50]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[51]  P. G. Ciarlet,et al.  Introduction to Numerical Linear Algebra and Optimisation , 1989 .

[52]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[53]  Sheng Chen,et al.  Representations of non-linear systems: the NARMAX model , 1989 .

[54]  Jia Zhang,et al.  Convergence and limit points of neural network and its application to pattern recognition , 1989, IEEE Trans. Syst. Man Cybern..

[55]  J. Shynk Adaptive IIR filtering , 1989, IEEE ASSP Magazine.

[56]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[57]  V. Gholkar Mean square convergence analysis of LMS algorithm (adaptive filters) , 1990 .

[58]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.

[59]  John J. Shynk,et al.  Convergence properties and stationary points of a perceptron learning algorithm , 1990 .

[60]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[61]  A. Khotanzad,et al.  Non-parametric prediction of AR processes using neural networks , 1990, International Conference on Acoustics, Speech, and Signal Processing.

[62]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[63]  Kurt Hornik,et al.  Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks , 1990, Neural Networks.

[64]  Russell C. Eberhart Standardization of neural network terminology , 1990, IEEE Trans. Neural Networks.

[65]  C.F.N. Cowan,et al.  Performance comparison of RLS and LMS algorithms for tracking a first order Markov communications channel , 1990, IEEE International Symposium on Circuits and Systems.

[66]  Neil E. Cotter,et al.  The Stone-Weierstrass theorem and its application to neural networks , 1990, IEEE Trans. Neural Networks.

[67]  Bernard Widrow,et al.  30 years of adaptive neural networks: perceptron, Madaline, and backpropagation , 1990, Proc. IEEE.

[68]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[69]  Thomas L. Clarke,et al.  Generalization of neural networks to the complex plane , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[70]  Jacek M. Zurada,et al.  Sufficient condition for convergence of a relaxation algorithm in actual single-layer neural networks , 1990, IEEE Trans. Neural Networks.

[71]  Heinz Unbehauen,et al.  Structure identification of nonlinear dynamic systems - A survey on input/output approaches , 1990, Autom..

[72]  Clark C. Guest,et al.  Modification of backpropagation networks for complex-valued signal processing in frequency domain , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[73]  L. Jones Constructive approximations for neural networks by sigmoidal functions , 1990, Proc. IEEE.

[74]  Les E. Atlas,et al.  Recurrent Networks and NARMA Modeling , 1991, NIPS.

[75]  C. N. Manikopoulos,et al.  Neural net nonlinear prediction for speech data , 1991 .

[76]  Kurt Hornik,et al.  Convergence of learning algorithms with constant learning rates , 1991, IEEE Trans. Neural Networks.

[77]  Jonathan J. Kaufman,et al.  Volterra characterization of neural networks , 1991, [1991] Conference Record of the Twenty-Fifth Asilomar Conference on Signals, Systems & Computers.

[78]  V. J. Mathews Adaptive polynomial filters , 1991, IEEE Signal Processing Magazine.

[79]  K. P. Unnikrishnan,et al.  Nonlinear prediction of speech signals using memory neuron networks , 1991, Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop.

[80]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[81]  Richard A. Davis,et al.  Time Series: Theory and Methods , 2013 .

[82]  Javier R. Movellan,et al.  Benefits of gain: speeded learning and minimal hidden layers in back-propagation networks , 1991, IEEE Trans. Syst. Man Cybern..

[83]  L. Personnaz,et al.  Neural network training schemes for non-linear adaptive filtering and modelling , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[84]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[85]  Visakan Kadirkamanathan,et al.  A nonlinear model for time series prediction and signal interpolation , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[86]  J. J. Shynk,et al.  Statistical analysis of the single-layer backpropagation algorithm , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[87]  B. Townshend,et al.  Nonlinear prediction of speech , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[88]  Mark E. Oxley,et al.  Comparative Analysis of Backpropagation and the Extended Kalman Filter for Training Multilayer Perceptrons , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[89]  Emile Fiesler,et al.  Neural Network Formalization , 1992 .

[90]  Cris Koutsougeras,et al.  Complex domain backpropagation , 1992 .

[91]  Peter J. Gawthrop,et al.  Neural networks for control systems - A survey , 1992, Autom..

[92]  L. K. Li Approximation theory and recurrent networks , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[93]  William A. Sethares,et al.  Adaptive algorithms with nonlinear data and error functions , 1992, IEEE Trans. Signal Process..

[94]  Shengwei Zhang,et al.  Lagrange programming neural networks , 1992 .

[95]  Ronald J. Williams,et al.  Training recurrent networks using the extended Kalman filter , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[96]  Markus Höhfeld,et al.  Learning with limited numerical precision using the cascade-correlation algorithm , 1992, IEEE Trans. Neural Networks.

[97]  A. Uncini Piazza,et al.  Artificial neural networks with adaptive polynomial activation function , 1992 .

[98]  S Z Qin,et al.  Comparison of four neural net learning methods for dynamic system identification , 1992, IEEE Trans. Neural Networks.

[99]  L. Ljung,et al.  A system identification perspective on neural nets , 1992, Neural Networks for Signal Processing II Proceedings of the 1992 IEEE Workshop.

[100]  Stephen A. Billings,et al.  Properties of neural networks with applications to modelling non-linear dynamical systems , 1992 .

[101]  Jhing-Fa Wang,et al.  A data-reuse architecture for gray-scale morphologic operations , 1992 .

[102]  Hideaki Sakai,et al.  A real-time learning algorithm for a multilayered neural network based on the extended Kalman filter , 1992, IEEE Trans. Signal Process..

[103]  Vera Kurková,et al.  Kolmogorov's theorem and multilayer neural networks , 1992, Neural Networks.

[104]  W. Wallace Galileo’s Logical Treatises , 1992 .

[105]  Giovanni Soda,et al.  Local Feedback Multilayered Networks , 1992, Neural Computation.

[106]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[107]  Francesco Piazza,et al.  On the complex backpropagation algorithm , 1992, IEEE Trans. Signal Process..

[108]  A. Tsoi,et al.  Implementation issues of sigmoid function and its derivative for VLSI digital neural networks , 1992 .

[109]  C. Micchelli,et al.  Approximation by superposition of sigmoidal and radial basis functions , 1992 .

[110]  C. P. Sheppard,et al.  Predicting time series by a fully connected neural network trained by back propagation , 1992 .

[111]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[112]  Henry Leung,et al.  Rational Function Neural Network , 1993, Neural Computation.

[113]  Pierre Roussel-Ragot,et al.  Neural Networks and Nonlinear Adaptive Filtering: Unifying Concepts and New Algorithms , 1993, Neural Computation.

[114]  Russell Reed,et al.  Pruning algorithms-a survey , 1993, IEEE Trans. Neural Networks.

[115]  M. M. Gupta,et al.  Dynamic neural unit and function approximation , 1993, IEEE International Conference on Neural Networks.

[116]  Andreas S. Weigend,et al.  The Future of Time Series: Learning and Understanding , 1993 .

[117]  Yuichi Nakamura,et al.  Approximation of dynamical systems by continuous time recurrent neural networks , 1993, Neural Networks.

[118]  Andrzej Cichocki,et al.  Neural networks for optimization and signal processing , 1993 .

[119]  W. K. Jenkins,et al.  New data-reusing LMS algorithms for improved convergence , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[120]  Michael T. Manry,et al.  Conventional modeling of the multilayer perceptron using polynomial basis functions , 1993, IEEE Trans. Neural Networks.

[121]  V. John Mathews,et al.  A stochastic gradient adaptive filter with gradient adaptive step size , 1993, IEEE Trans. Signal Process..

[122]  Luigi Fortuna,et al.  On the capability of neural networks with complex neurons in complex valued functions approximation , 1993, 1993 IEEE International Symposium on Circuits and Systems.

[123]  Francesco Piazza,et al.  Neural networks with digital LUT activation functions , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[124]  Eric A. Wan,et al.  Time series prediction by using a connectionist network with internal delay lines , 1993 .

[125]  John J. Shynk,et al.  Statistical analysis of the single-layer backpropagation algorithm. II. MSE and classification performance , 1993, IEEE Trans. Signal Process..

[126]  Kurt Hornik,et al.  Some new results on neural network approximation , 1993, Neural Networks.

[127]  David A. Sprecher,et al.  A universal mapping for kolmogorov's superposition theorem , 1993, Neural Networks.

[128]  M. Mansour,et al.  Stability of polynomials with time-variant coefficients , 1993 .

[129]  James Theiler,et al.  Detecting Nonlinearity in Data with Long Coherence Times , 1993, comp-gas/9302003.

[130]  M. Sarhadi,et al.  Performance aspects of a novel neuron activation function in multi-layer feed-forward networks , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[131]  M. Sarhadi,et al.  A modified neuron activation function which enables single layer perceptrons to solve some linearly inseparable problems , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[132]  S. Haykin,et al.  A cascaded recurrent neural network for real-time nonlinear adaptive filtering , 1993, IEEE International Conference on Neural Networks.

[133]  Allan Pinkus,et al.  Multilayer Feedforward Networks with a Non-Polynomial Activation Function Can Approximate Any Function , 1991, Neural Networks.

[134]  José Carlos Príncipe,et al.  The gamma-filter-a new class of adaptive IIR filters with restricted feedback , 1993, IEEE Trans. Signal Process..

[135]  David L. Elliott,et al.  A Better Activation Function for Artificial Neural Networks , 1993 .

[136]  Mahesan Niranjan,et al.  A dynamic neural network architecture by sequential partitioning of the input space , 1993, IEEE International Conference on Neural Networks.

[137]  James Ting-Ho Lo,et al.  Synthetic approach to optimal filtering , 1994, IEEE Trans. Neural Networks.

[138]  Liang Jin,et al.  Absolute stability conditions for discrete-time recurrent neural networks , 1994, IEEE Trans. Neural Networks.

[139]  Lizhong Wu,et al.  On the design of nonlinear speech predictors with recurrent nets , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[140]  Hidefumi Katsuura,et al.  Computational aspects of Kolmogorov's superposition theorem , 1994, Neural Networks.

[141]  Donald E. Waagen,et al.  Evolving recurrent perceptrons for time-series modeling , 1994, IEEE Trans. Neural Networks.

[142]  P. Regalia Adaptive IIR Filtering in Signal Processing and Control , 1994 .

[143]  A. Tesi,et al.  Conditions for global stability of some classes of nonsymmetric neural networks , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[144]  Andreas S. Weigend,et al.  Time Series Prediction: Forecasting the Future and Understanding the Past , 1994 .

[145]  Xuan Kong,et al.  Adaptive Signal Processing Algorithms: Stability and Performance , 1994 .

[146]  Amir F. Atiya,et al.  How delays affect neural dynamics and learning , 1994, IEEE Trans. Neural Networks.

[147]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[148]  Pierre Roussel-Ragot,et al.  Training recurrent neural networks: why and how? An illustration in dynamical process modeling , 1994, IEEE Trans. Neural Networks.

[149]  S. Douglas A family of normalized LMS algorithms , 1994, IEEE Signal Processing Letters.

[150]  Les E. Atlas,et al.  Recurrent neural networks and robust time series prediction , 1994, IEEE Trans. Neural Networks.

[151]  Martin Brown,et al.  Aspects of instantaneous on-line learning rules , 1994 .

[152]  Uwe Helmke,et al.  Neural networks, rational functions, and realization theory , 1995, Math. Control. Signals Syst..

[153]  Uwe Helmke,et al.  Existence and uniqueness results for neural network approximations , 1995, IEEE Trans. Neural Networks.

[154]  Ronald J. Williams,et al.  Gradient-based learning algorithms for recurrent networks and their computational complexity , 1995 .

[155]  M. Mansour,et al.  Robust stability of time-variant discrete-time systems with bounded parameter perturbations , 1995 .

[156]  G. U. Yule,et al.  The Foundations of Econometric Analysis: On a Method of Investigating Periodicities in Disturbed Series, with Special Reference to Wolfer's Sunspot Numbers ( Philosophical Transactions of the Royal Society of London , A, vol. 226, 1927, pp. 267–73) , 1995 .

[157]  Scott C. Douglas,et al.  Exact expectation analysis of the LMS adaptive filter , 1995, IEEE Trans. Signal Process..

[158]  Tamer Başar,et al.  H1-Optimal Control and Related Minimax Design Problems , 1995 .

[159]  L. Ljung,et al.  Exponential stability of general tracking algorithms , 1995, IEEE Trans. Autom. Control..

[160]  William Fornaciari,et al.  Behavior-driven minimal implementation of digital ANNs , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[161]  Eduardo Sontag,et al.  Computational power of neural networks , 1995 .

[162]  Marios M. Polycarpou,et al.  High-order neural network structures for identification of dynamical systems , 1995, IEEE Trans. Neural Networks.

[163]  Liang Li,et al.  Nonlinear adaptive prediction of nonstationary signals , 1995, IEEE Trans. Signal Process..

[164]  R. Pearson Nonlinear Input/Output Modeling , 1994 .

[165]  John H. Mathews,et al.  Complex analysis for mathematics and engineering , 1995 .

[166]  Kazuo Tanaka,et al.  Stability analysis of neural networks via Lyapunov approach , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[167]  Sailes K. Sengijpta Fundamentals of Statistical Signal Processing: Estimation Theory , 1995 .

[168]  Shaun-Inn Wu Mirroring our thought processes [recurrent neural network and time series in forecasting] , 1996 .

[169]  Vincenzo Piuri,et al.  Experimental neural networks for prediction and identification , 1996 .

[170]  S. Zakeri On Critical Points of Proper Holomorphic Maps on The Unit Disk , 1996, math/9606221.

[171]  Masahiro Nakagawa,et al.  An autonomously controlled chaos neural network , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[172]  Emile Fiesler,et al.  The Interchangeability of Learning Rate and Gain in Backpropagation Neural Networks , 1996, Neural Computation.

[173]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[174]  K. S. Narendra,et al.  Neural networks for control theory and practice , 1996, Proc. IEEE.

[175]  M. Glesner,et al.  Searching for robust chaos in discrete time neural networks using weight space exploration , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[176]  Gaetan Libert,et al.  Dynamic recurrent neural networks: a dynamical analysis , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[177]  Liang Jin,et al.  Globally asymptotical stability of discrete-time analog neural networks , 1996, IEEE Trans. Neural Networks.

[178]  Simon Haykin,et al.  Neural networks expand SP's horizons , 1996, IEEE Signal Process. Mag..

[179]  Yoshua Bengio,et al.  Neural networks for speech and sequence recognition , 1996 .

[180]  B. Kosko,et al.  What is the best shape for a fuzzy set in function approximation? , 1996, Proceedings of IEEE 5th International Fuzzy Systems.

[181]  Russell Beale,et al.  Handbook of Neural Computation , 1996 .

[182]  Peter Tiño,et al.  Learning long-term dependencies in NARX recurrent neural networks , 1996, IEEE Trans. Neural Networks.

[183]  Francesco Piazza,et al.  Approximation capabilities of adaptive spline neural networks , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[184]  Hava T. Siegelmann,et al.  Computational capabilities of recurrent NARX neural networks , 1997, IEEE Trans. Syst. Man Cybern. Part B.

[185]  Emile Fiesler,et al.  Optimal Setting of Weights, Learning Rate, and Gain , 1997 .

[186]  V. Lazarescu,et al.  On viewing the transform performed by a hidden layer in a feedforward ANN, as a complex Mobius mapping , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[187]  Scott C. Douglas,et al.  Adaptive filters employing partial updates , 1997 .

[188]  Toru Yamaguchi,et al.  Necessary and Sufficient Condition for Absolute Exponential Stability of a Class of Nonsymmetric Neural Networks , 1997 .

[189]  Sun-Yuan Kung,et al.  A delay damage model selection algorithm for NARX neural networks , 1997, IEEE Trans. Signal Process..

[190]  A. Tikhonov,et al.  Nonlinear Ill-Posed Problems , 1997 .

[191]  Gilles Pagès,et al.  Approximations of Functions by a Multilayer Perceptron: a New Approach , 1997, Neural Networks.

[192]  K. Aihara,et al.  Chaos and asymptotical stability in discrete-time neural networks , 1997, chao-dyn/9701020.

[193]  Bart Kosko,et al.  Adaptive joint fuzzy sets for function approximation , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[194]  Po-Rong Chang,et al.  Optimal Nonlinear Adaptive Prediction and Modeling of MPEG Video in ATM Networks Using Pipelined Recurrent Neural Networks , 1997, IEEE J. Sel. Areas Commun..

[195]  K. M. Al-Ruwaihi CMOS analogue neurone circuit with programmable activation functions utilising MOS transistors with optimised process/device parameters , 1997 .

[196]  Emile Fiesler,et al.  High-order and multilayer perceptron initialization , 1997, IEEE Trans. Neural Networks.

[197]  R. de Figueiredo Optimal neural network realizations of nonlinear FIR and IIR filters , 1997, Proceedings of 1997 IEEE International Symposium on Circuits and Systems. Circuits and Systems in the Information Age ISCAS '97.

[198]  R.J. Cohen,et al.  Linear and nonlinear ARMA model parameter estimation using an artificial neural network , 1997, IEEE Transactions on Biomedical Engineering.

[199]  L. Ljung,et al.  Necessary and sufficient conditions for stability of LMS , 1997, IEEE Trans. Autom. Control..

[200]  Sandro Ridella,et al.  Circular backpropagation networks for classification , 1997, IEEE Trans. Neural Networks.

[201]  Jose C. Principe,et al.  The past, present, and future of neural networks for signal processing , 1997 .

[202]  James R. Zeidler,et al.  Adaptive tracking of linear time-variant systems by extended RLS algorithms , 1997, IEEE Trans. Signal Process..

[203]  S. Douglas,et al.  A posteriori updates for adaptive filters , 1997, Conference Record of the Thirty-First Asilomar Conference on Signals, Systems and Computers (Cat. No.97CB36136).

[204]  Rafik A. Goubran,et al.  Nonlinear adaptive filtering with FIR synapses and adaptive activation functions , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[205]  Michael I. Jordan Serial Order: A Parallel Distributed Processing Approach , 1997 .

[206]  Yu Wei,et al.  Combining adaptive sigmoid packet and trace neural network for fast invariance-learning , 1998 .

[207]  C A Bailer-Jones,et al.  A recurrent neural network for modelling dynamical systems. , 1998, Network.

[208]  Sung-Suk Kim Time-delay recurrent neural network for temporal correlations and prediction , 1998, Neurocomputing.

[209]  Danilo P. Mandic,et al.  A posteriori real-time recurrent learning schemes for a recurrent neural network based nonlinear predictor , 1998 .

[210]  Danilo P. Mandic,et al.  Advanced PRNN based nonlinear prediction/system identification , 1998 .

[211]  I. Hlavácek,et al.  A posteriori error estimates for three-dimensional axisymmetric elliptic problems , 1998 .

[212]  David A. Medler A Brief History of Connectionism , 1998 .

[213]  K. Rose Deterministic annealing for clustering, compression, classification, regression, and related optimization problems , 1998, Proc. IEEE.

[214]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[215]  Jonathon A. Chambers,et al.  From an a priori RNN to an a posteriori PRNN nonlinear predictor , 1998, Neural Networks for Signal Processing VIII. Proceedings of the 1998 IEEE Signal Processing Society Workshop (Cat. No.98TH8378).

[216]  Danilo P. Mandic,et al.  Non-Linear Adaptive Prediction of Speech with a Pipelined Recurrent Neural Network and Advanced Learning Algorithms , 1998 .

[217]  K. Nakayama,et al.  A Cascade Form Predictor of Neural and FIR Filters and Its Minimum Size Estimation Based on Nonlinearity Analysis of Time Series , 1998 .

[218]  P. Arena,et al.  Neural Networks in Multidimensional Domains: Fundamentals and New Trends in Modelling and Control , 1998 .

[219]  Alexander N Gorban,et al.  The general approximation theorem , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[220]  Jacques Sombrin,et al.  Neural network modeling and identification of nonlinear channels with memory: algorithms, applications, and analytic models , 1998, IEEE Trans. Signal Process..

[221]  Gérard Dreyfus,et al.  Comment on "Discrete-time recurrent neural network architectures: A unifying review" , 1998, Neurocomputing.

[222]  Jenq-Neng Hwang,et al.  Neural networks for intelligent multimedia processing , 1998 .

[223]  B. Ermentrout Neural networks as spatio-temporal pattern-forming systems , 1998 .

[224]  Yizhak Idan,et al.  The Canonical Form of Nonlinear Discrete-Time Models , 1998, Neural Computation.

[225]  Yong Soo Cho,et al.  Adaptive precompensation of Wiener systems , 1998, IEEE Trans. Signal Process..

[226]  Majid Ahmadi,et al.  Model validation and determination for neural network activation function modeling , 1998, 1998 Midwest Symposium on Circuits and Systems (Cat. No. 98CB36268).

[227]  Javier Calpe-Maravilla,et al.  An easy demonstration of the optimum value of the adaptation constant in the LMS algorithm [FIR filt , 1998 .

[228]  Valeriu Beiu,et al.  On Kolmogorov's superpositions and Boolean functions , 1998, Proceedings 5th Brazilian Symposium on Neural Networks (Cat. No.98EX209).

[229]  Danilo P. Mandic,et al.  Toward an optimal PRNN-based nonlinear predictor , 1999, IEEE Trans. Neural Networks.

[230]  Andrew Chi-Sing Leung,et al.  On the Kalman filtering method in neural network training and pruning , 1999, IEEE Trans. Neural Networks.

[231]  R. Devaney The Mandelbrot Set, the Farey Tree, and the Fibonacci Sequence , 1999 .

[232]  Danilo P. Mandic,et al.  Global asymptotic convergence of nonlinear relaxation equations realised through a recurrent perceptron , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[233]  J. A. Chambers,et al.  Exploiting inherent relationships in RNN architectures , 1999, Neural Networks.

[234]  Allan Pinkus,et al.  Lower bounds for approximation by MLP neural networks , 1999, Neurocomputing.

[235]  S. Haykin,et al.  Lessons on adaptive systems for signal processing, communications, and control , 1999, IEEE Signal Processing Magazine.

[236]  Norbert Jankowski,et al.  Survey of Neural Transfer Functions , 1999 .

[237]  Kenji Nakayama,et al.  A Hybrid Nonlinear Predictor: Analysis of Learning Process and Predictability for Noisy Time Series (Special Section on Digital Signal Processing) , 1999 .

[238]  Danilo P. Mandic,et al.  A Nonlinear Adaptive Predictor Realised Via Recurrent Neural Networks With Annealing , 1999 .

[239]  Francesco Piazza,et al.  Multilayer feedforward networks with adaptive spline activation function , 1999, IEEE Trans. Neural Networks.

[240]  Simon Haykin,et al.  A dynamic regularized radial basis function network for nonlinear, nonstationary time series prediction , 1999, IEEE Trans. Signal Process..

[241]  Alan V. Oppenheim,et al.  Discrete-time signal processing (2nd ed.) , 1999 .

[242]  Persi Diaconis,et al.  Iterated Random Functions , 1999, SIAM Rev..

[243]  Danilo P. Mandic,et al.  A posteriori error learning in nonlinear adaptive filters , 1999 .

[244]  Paul C. Kainen,et al.  Approximation by neural networks is not continuous , 1999, Neurocomputing.

[245]  Kurt Kreith,et al.  Iterative Algebra and Dynamic Modeling: A Curriculum for the Third Millennium , 1999 .

[246]  F. Becker,et al.  Detecting nonlinearities in time series of machining processes , 1999, Proceedings of the 1999 American Control Conference (Cat. No. 99CH36251).

[247]  Thomas Elsken,et al.  Smaller nets may perform better: special transfer functions , 1999, Neural Networks.

[248]  Igor R. Krcmar,et al.  Global asymptotic stability for RNNs with a bipolar activation function , 2000, Proceedings of the 5th Seminar on Neural Network Applications in Electrical Engineering. NEUREL 2000 (IEEE Cat. No.00EX287).

[249]  Danilo P. Mandic,et al.  On global asymptotic stability of fully connected recurrent neural networks , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[250]  D. Mandic,et al.  On stability of relaxive systems described by polynomials with time-variant coefficients , 2000 .

[251]  John N. Tsitsiklis,et al.  A survey of computational complexity results in systems and control , 2000, Autom..

[252]  D. P. Mandic,et al.  NNGD algorithm for neural adaptive filters , 2000 .

[253]  Eduardo D. Sontag,et al.  Neural Systems as Nonlinear Filters , 2000, Neural Computation.

[254]  Danilo P. Mandic,et al.  A normalized gradient algorithm for an adaptive recurrent perceptron , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[255]  Misha Tsodyks,et al.  Chaos in neural networks with dynamic synapses , 2000, Neurocomputing.

[256]  Peter B. Luh,et al.  Lagrangian relaxation neural networks for job shop scheduling , 2000, IEEE Trans. Robotics Autom..

[257]  Danilo P. Mandic,et al.  Relationships Between the A Priori and A Posteriori Errors in Nonlinear Adaptive Neural Filters , 2000, Neural Computation.

[258]  Danilo P. Mandic,et al.  Advanced RNN Based NARMA Predictors , 2000, J. VLSI Signal Process..

[259]  S. Haykin Unsupervised adaptive filtering, vol. 1: Blind source separation , 2000 .

[260]  Danilo P. Mandic,et al.  On fixed points of a general neural network via Mobius transformations , 2000 .

[261]  Hung T. Nguyen,et al.  Fuzzy systems are universal approximators for a smooth function and its derivatives , 2000 .

[262]  D. Mandic The use of Mobius transformations in neural networks and signal processing , 2000, Neural Networks for Signal Processing X. Proceedings of the 2000 IEEE Signal Processing Society Workshop (Cat. No.00TH8501).

[263]  Danilo P. Mandic,et al.  A normalised real time recurrent learning algorithm , 2000, Signal Process..

[264]  Danilo P. Mandic,et al.  On training with slope adaptation for feedforward NNs , 2000, Proceedings of the 5th Seminar on Neural Network Applications in Electrical Engineering. NEUREL 2000 (IEEE Cat. No.00EX287).

[265]  D. Mandic,et al.  On robust stability of time-variant discrete-time nonlinear systems with bounded parameter perturbations , 2000 .

[266]  D. Mandic,et al.  Stability of NNGD algorithm for nonlinear system identification , 2001 .

[267]  Gavin C. Cawley,et al.  On Nonlinear Processing of Air Pollution Data , 2001 .

[268]  G. Lewicki,et al.  Approximation by Superpositions of a Sigmoidal Function , 2003 .