Application of Generalized Dynamic Neural Networks to Biomedical Data

This paper reviews the application of continuous recurrent neural networks with time-varying weights to pattern recognition tasks in medicine. A general learning algorithm based on Pontryagin's maximum principle is recapitulated, and possibilities of improving the generalization capabilities of these networks are given. The effectiveness of the methods is demonstrated by three different real-world examples taken from the fields of anesthesiology, orthopedics, and radiology

[1]  N R Webster,et al.  Auditory evoked response and awareness: a study in volunteers at sub-MAC concentrations of isoflurane. , 1992, British journal of anaesthesia.

[2]  C. Doré,et al.  Somatosensory and auditory evoked responses recorded simultaneously: differential effects of nitrous oxide and isoflurane. , 1992, British journal of anaesthesia.

[3]  Herbert Witte,et al.  Improved learning of multiple continuous trajectories with initial network state , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[4]  Herbert Witte,et al.  Classification of equinus in ambulatory children with cerebral palsy-discrimination between dynamic tightness and fixed contracture. , 2004, Gait & posture.

[5]  Malur K. Sundareshan,et al.  Periodic motions, mapping ordered sequences, and training of dynamic neural networks to generate continuous and discontinuous trajectories , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[6]  Keinosuke Fukunaga,et al.  A Branch and Bound Algorithm for Feature Subset Selection , 1977, IEEE Transactions on Computers.

[7]  L. Leistritz,et al.  Prediction of movement following noxious stimulation during 1 minimum alveolar anesthetic concentration isoflurane/nitrous oxide anesthesia by means of middle latency auditory evoked responses , 2002, Clinical Neurophysiology.

[8]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[9]  S. Õunpuu,et al.  Alterations in surgical decision making in patients with cerebral palsy based on three-dimensional gait analysis. , 1997, Journal of pediatric orthopedics.

[10]  Warren D. Smith,et al.  Measuring the Performance of Anesthetic Depth Indicators , 1996, Anesthesiology.

[11]  Kwang Y. Lee,et al.  Diagonal recurrent neural networks for dynamic systems control , 1995, IEEE Trans. Neural Networks.

[12]  Herbert Witte,et al.  Improving Generalization Capabilities of Dynamic Neural Networks , 2004, Neural Computation.

[13]  H. Witte,et al.  Identification of the stimulated hemiretina in primary school children and adults based on left and right hemifield pattern reversal visual evoked potentials – a comparative study , 2001, Clinical Neurophysiology.

[14]  E. Pöppel,et al.  Effects of surgical stimulation on midlatency auditory evoked potentials during general anaesthesia with propofol/fentanyl, isoflurane/fentanyl and flunitrazepam/fentanyl , 1994, Anaesthesia.

[15]  Stanley Tuhrim,et al.  Antithrombotic trials in acute ischaemic stroke: a selective review , 2004, Expert opinion on emerging drugs.

[16]  Pedro J. Zufiria,et al.  Time-Scaling in Recurrent Neural Learning , 2002, ICANN.

[17]  E Konecny,et al.  Middle latency auditory evoked responses and electroencephalographic derived variables do not predict movement to noxious stimulation during 1 minimum alveolar anesthetic concentration isoflurane/nitrous oxide anesthesia. , 1999, Anesthesia and analgesia.

[18]  Danil V. Prokhorov,et al.  Simple and conditioned adaptive behavior from Kalman filter trained recurrent networks , 2003, Neural Networks.

[19]  J. Barhen,et al.  Adjoint-operators and non-adiabatic learning algorithms in neural networks , 1991 .

[20]  A. PearlmutterB..  Gradient calculations for dynamic recurrent neural networks , 1995 .

[21]  G. V. Puskorius,et al.  A signal processing framework based on dynamic neural networks with application to problems in adaptation, filtering, and classification , 1998, Proc. IEEE.

[22]  C J Kalkman,et al.  Quality of perioperative AEP--variability of expert ratings. , 2003, British journal of anaesthesia.

[23]  Herbert Witte,et al.  Learning continuous trajectories in recurrent neural networks with time-dependent weights , 1999, IEEE Trans. Neural Networks.

[24]  Herbert Witte,et al.  Training trajectories by continuous recurrent multilayer networks , 2002, IEEE Trans. Neural Networks.

[25]  Barak A. Pearlmutter.  Gradient calculations for dynamic recurrent neural networks: a survey , 1995, IEEE Trans. Neural Networks.

[26]  Udilo Finsterer,et al.  The Effects of Anesthesia with Increasing End-Expiratory Concentrations of Sevoflurane on Midlatency Auditory Evoked Potentials , 1995, Anesthesia and analgesia.

[27]  Ravi Kothari,et al.  Feature subset selection using a new definition of classifiability , 2003, Pattern Recognit. Lett..

[28]  Ernst Pöppel,et al.  Mid-latency auditory evoked potentials and circulatory response to loud sounds , 1994 .

[29]  P J Webros.  BACKPROPAGATION THROUGH TIME: WHAT IT DOES AND HOW TO DO IT , 1990 .

[30]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[31]  C. Beaulieu,et al.  Longitudinal magnetic resonance imaging study of perfusion and diffusion in stroke: Evolution of lesion volume and correlation with clinical outcome , 1999, Annals of neurology.

[32]  Amir F. Atiya,et al.  Application of the recurrent multilayer perceptron in modeling complex process dynamics , 1994, IEEE Trans. Neural Networks.

[33]  R. V. Dooren,et al.  A Chebyshev technique for solving nonlinear optimal control problems , 1988 .

[34]  Robert Tibshirani,et al.  A Comparison of Some Error Estimates for Neural Network Models , 1996, Neural Computation.

[35]  D. Antonelli,et al.  Gait analysis of the triceps surae in cerebral palsy. A preoperative and postoperative clinical and electromyographic study. , 1974, The Journal of bone and joint surgery. American volume.

[36]  E Pöppel,et al.  Midlatency auditory evoked potentials during anaesthesia with increasing endexpiratory concentrations of desflurane , 1996, Acta anaesthesiologica Scandinavica.

[37]  R J Seitz,et al.  Diffusion- and perfusion-weighted MRI. The DWI/PWI mismatch region in acute stroke. , 1999, Stroke.

[38]  E Pöppel,et al.  Motor signs of wakefulness during general anaesthesia with propofol, isoflurane and flunitrazepam/fentanyl and midlatency auditory evoked potentials , 1994, Anaesthesia.

[39]  Anthony J. Robinson,et al.  Static and Dynamic Error Propagation Networks with Application to Speech Coding , 1987, NIPS.

[40]  Yuichi Nakamura,et al.  Approximation of dynamical systems by continuous time recurrent neural networks , 1993, Neural Networks.

[41]  E. Pöppel,et al.  Sensory information processing during general anaesthesia: effect of isoflurane on auditory evoked neuronal oscillations. , 1991, British journal of anaesthesia.

[42]  Tomaso A. Poggio,et al.  Regularization Theory and Neural Networks Architectures , 1995, Neural Computation.

[43]  G. Schlaug,et al.  The ischemic penumbra: operationally defined by diffusion and perfusion MRI. , 1999, Neurology.

[44]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[45]  Herbert Witte,et al.  Structure optimization of neural networks with the A*-algorithm , 1997, IEEE Trans. Neural Networks.

[46]  B. Etnyre,et al.  Preoperative and postoperative assessment of surgical intervention for equinus gait in children with cerebral palsy. , 1993, Journal of pediatric orthopedics.

[47]  H. Witte,et al.  Identification of hemifield single trial PVEP on the basis of generalized dynamic neural network classifiers , 1999, Clinical Neurophysiology.

[48]  Jochen J. Steil.  Local stability of recurrent networks with time-varying weights and inputs , 2002, Neurocomputing.

[49]  Barak A. Pearlmutter.  Learning State Space Trajectories in Recurrent Neural Networks , 1989, Neural Computation.

[50]  D. V. Sridhar,et al.  Information theoretic subset selection for neural network models , 1998 .

[51]  J. Gallego,et al.  INTRODUCTION TO OPTIMAL CONTROL THEORY , 1985 .

[52]  Lee A. Feldkamp,et al.  Neurocontrol of nonlinear dynamical systems with Kalman filter trained recurrent networks , 1994, IEEE Trans. Neural Networks.

[53]  Kwang Y. Lee,et al.  An optimal tracking neuro-controller for nonlinear dynamic systems , 1996, IEEE Trans. Neural Networks.

[54]  G Stockmanns,et al.  Wavelet Analysis of Middle Latency Auditory Evoked Responses: Calculation of an Index for Detection of Awareness during Propofol Administration , 2001, Anesthesiology.

[55]  Jacques Ludik,et al.  A Multilayer Real-Time Recurrent Learning Algorithm for Improved Convergence , 1997, ICANN.

[56]  Marios M. Polycarpou,et al.  High-order neural network structures for identification of dynamical systems , 1995, IEEE Trans. Neural Networks.

[57]  Pierre Baldi,et al.  Gradient descent learning algorithm overview: a general dynamical systems perspective , 1995, IEEE Trans. Neural Networks.

[58]  Raúl Rojas,et al.  Neural Networks - A Systematic Introduction , 1996 .

[59]  Chuanyi Ji,et al.  Fast training of recurrent networks based on the EM algorithm , 1998, IEEE Trans. Neural Networks.

[60]  S. Õunpuu,et al.  Kinematic and Kinetic Evaluation of the Ankle After Lengthening of the Gastrocnemius Fascia in Children with Cerebral Palsy , 1993, Journal of pediatric orthopedics.

[61]  Ewald Konecny,et al.  INTEROBSERVER-VARIABILITY FOR EVALUATION OF MIDDLE LATENCY AUDITORY EVOKED POTENTIALS DURING ANESTHESIA , 1998 .

[62]  E. Zwick,et al.  Fixed and Dynamic Equinus in Cerebral Palsy: Evaluation of Ankle Function After Multilevel Surgery , 2001, Journal of pediatric orthopedics.

[63]  David J. Hand,et al.  Discrimination and Classification , 1982 .

[64]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[65]  Emanuel Marom,et al.  Efficient Training of Recurrent Neural Network with Time Delays , 1997, Neural Networks.

[66]  Visakan Kadirkamanathan,et al.  Dynamic structure neural networks for stable adaptive control of nonlinear systems , 1996, IEEE Trans. Neural Networks.

[67]  Kenji Doya,et al.  Universality of Fully-Connected Recurrent Neural Networks , 1993 .

[68]  B. Rosen,et al.  Perfusion imaging with NMR contrast agents , 1990, Magnetic resonance in medicine.