Neural networks for automatic target recognition

Abstract Many applications reported in artificial neural networks are associated with military problems. This paper reviews concepts associated with the processing of military data to find and recognize targets—automatic target recognition (ATR). A general-purpose automatic target recognition system does not exist. The work presented here is demonstrated on military data, but it can only be consideredproof of principle until systems are fielded andproven “under-fire”. ATR data can be in the form of non-imaging one-dimensional sensor returns, such as ultra-high range-resolution radar returns for air-to-air automatic target recognition and vibration signatures from a laser radar for recognition of ground targets. The ATR data can be two-dimensional images. The most common ATR images are infrared, but current systems must also deal with synthetic aperture radar images. Finally, the data can be three-dimensional, such as sequences of multiple exposures taken over time from a nonstationary world. Targets move, as do sensors, and that movement can be exploited by the ATR. Hyperspectral data, which are views of the same piece of the world looking at different spectral bands, is another example of multiple image data; the third dimension is now wavelength and not time. ATR system design usually consists of four stages. The first stage is to select the sensor or sensors to produce the target measurements. The next stage is the preprocessing of the data and the location of regions of interest within the data (segmentation). The human retina is a ruthless preprocessor. Physiology motivated preprocessing and segmentation is demonstrated along with supervised and unsupervised artificial neural segmentation techniques. The third design step is feature extraction and selection: the extraction of a set of numbers which characterize regions of the data. The last step is the processing of the features for decision making (classification). The area of classification is where most ATR related neural network research has been accomplished. The relation of neural classifiers to Bayesian techniques is emphasized along with the more recent use of feature sequences to enhance classification. The principal theme of this paper is that artificial neural networks have proven to be an interesting and useful alternate processing strategy. Artificial neural techniques, however, are not magical solutions with mystical abilities that work without good engineering. Good understanding of the capabilities and limitations of neural techniques is required to apply them productively to ATR problems.

[1]  Keinosuke Fukunaga,et al.  Bayes Error Estimation Using Parzen and k-NN Procedures , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Harvey F. Silverman,et al.  Combining hidden Markov model and neural network classifiers , 1990, International Conference on Acoustics, Speech, and Signal Processing.

[3]  D. Ruck,et al.  Spatio-temporal pattern recognition using hidden Markov models , 1995, IEEE Transactions on Aerospace and Electronic Systems.

[4]  Bruce W. Suter,et al.  The multilayer perceptron as an approximation to a Bayes optimal discriminant function , 1990, IEEE Trans. Neural Networks.

[5]  R. D. Lorenz,et al.  A structure by which a recurrent neural network can approximate a nonlinear dynamic system , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[6]  Steven K. Rogers,et al.  Bayesian selection of important features for feedforward neural networks , 1993, Neurocomputing.

[7]  Jean M Steppe Feature and Model Selection in Feedforward Neural Networks , 1994 .

[8]  James C Gainey Predicting Nonlinear Time Series , 1993 .

[9]  E. Adelson,et al.  Phenomenal coherence of moving visual patterns , 1982, Nature.

[10]  T Poggio,et al.  Regularization Algorithms for Learning That Are Equivalent to Multilayer Networks , 1990, Science.

[11]  I. Daubechies Orthonormal bases of compactly supported wavelets , 1988 .

[12]  Judith E. Dayhoff,et al.  Spatiotemporal topology and temporal sequence identification with an adaptive time-delay neural network , 1993, Other Conferences.

[13]  David Zipser,et al.  A Subgrouping Strategy that Reduces Complexity and Speeds Up Learning in Recurrent Networks , 1989, Neural Computation.

[14]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[15]  Stephen Grossberg,et al.  Neural dynamics of surface perception: Boundary webs, illuminants, and shape-from-shading , 1987, Comput. Vis. Graph. Image Process..

[16]  Eric A. Wan,et al.  Neural network classification: a Bayesian interpretation , 1990, IEEE Trans. Neural Networks.

[17]  Shigeki Miyake,et al.  Bayes statistical behavior and valid generalization of pattern classifying neural networks , 1991, IEEE Trans. Neural Networks.

[18]  Bernard Widrow,et al.  Adaptive switching circuits , 1988 .

[19]  Peter S. Maybeck,et al.  Application of sequence comparison techniques to multisensor data fusion and target recognition , 1993, Proceedings of 32nd IEEE Conference on Decision and Control.

[20]  Stéphane Mallat,et al.  Multifrequency Channel Decompositions of Images , 1989 .

[21]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[22]  A. A. Mullin,et al.  Principles of neurodynamics , 1962 .

[23]  Eric A. Wan,et al.  Time series prediction by using a connectionist network with internal delay lines , 1993 .

[24]  G. Kane Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol 1: Foundations, vol 2: Psychological and Biological Models , 1994 .

[25]  J. Robson Spatial and Temporal Contrast-Sensitivity Functions of the Visual System , 1966 .

[26]  G. L. Tarr,et al.  Multi-layered feedforward neural networks for image segmentation , 1992 .

[27]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[28]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[29]  Andrew B. Watson,et al.  A look at motion in the frequency domain , 1983 .

[30]  John S. Bridle,et al.  Alpha-nets: A recurrent 'neural' network architecture with a hidden Markov model interpretation , 1990, Speech Commun..

[31]  C. Mead,et al.  Neuromorphic Robot Vision with Mixed Analog- Digital Architecture , 2005 .

[32]  Hyuk Lee Discrete Synchronous Neural Algorithm for Minimization , 1989, Neural Computation.

[33]  David A. Landgrebe,et al.  Feature Extraction Based on Decision Boundaries , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[35]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[36]  S. McKee A local mechanism for differential velocity detection , 1981, Vision Research.

[37]  Patrick A. Shoemaker,et al.  A note on least-squares learning procedures and classification by neural network models , 1991, IEEE Trans. Neural Networks.

[38]  Mikio Takagi,et al.  Internal representation of a neural network that detects local motion , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[39]  Geoffrey E. Hinton,et al.  Phoneme recognition using time-delay neural networks , 1989, IEEE Trans. Acoust. Speech Signal Process..

[40]  Haluk Ögmen,et al.  Neural network architectures for motion perception and elementary motion detection in the fly visual system , 1990, Neural Networks.

[41]  J J Hopfield,et al.  Neural computation by concentrating information in time. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[42]  Richard Lippmann,et al.  Neural Network Classifiers Estimate Bayesian a posteriori Probabilities , 1991, Neural Computation.

[43]  L. Rabiner,et al.  An introduction to hidden Markov models , 1986, IEEE ASSP Magazine.

[44]  J. P. Jones,et al.  The two-dimensional spatial structure of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[45]  D. Ruck,et al.  A wavelet multiresolution analysis for spatio-temporal signals , 1996, IEEE Transactions on Aerospace and Electronic Systems.

[46]  Thomas J. Jackson,et al.  The Multilayer Perceptron , 1990 .

[47]  Pineda,et al.  Generalization of back-propagation to recurrent neural networks. , 1987, Physical review letters.

[48]  Fernando J. Pineda,et al.  Recurrent Backpropagation and the Dynamical Approach to Adaptive Neural Computation , 1989, Neural Computation.

[49]  Nabil H. Farhat Microwave diversity imaging and automated target identification based on models of neural networks , 1989 .

[50]  Stéphane Mallat,et al.  Multifrequency channel decompositions of images and wavelet models , 1989, IEEE Trans. Acoust. Speech Signal Process..

[51]  Athanasios Kehagias Stochastic Recurrent Networks: Prediction and Classification of Time Series , 1991 .

[52]  Steven Kay,et al.  Modern Spectral Estimation: Theory and Application , 1988 .

[53]  Stephen Grossberg,et al.  Invariant recognition of cluttered scenes by a self-organizing ART architecture: CORT-X boundary segmentation , 1989, Neural Networks.

[54]  S. Zeki A vision of the brain , 1993 .

[55]  Esther Levin,et al.  Word recognition using hidden control neural architecture , 1990, International Conference on Acoustics, Speech, and Signal Processing.

[56]  Godfried T. Toussaint,et al.  Bibliography on estimation of misclassification , 1974, IEEE Trans. Inf. Theory.

[57]  Misha Mahowald,et al.  A silicon model of early visual processing , 1993, Neural Networks.

[58]  Steven K. Rogers,et al.  Autonomous target recognition using remotely sensed surface vibration measurements , 1993, Defense, Security, and Sensing.

[59]  Steven K. Rogers,et al.  Neural network Bayes error estimation , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[60]  Yann LeCun,et al.  Optimal Brain Damage , 1989, NIPS.

[61]  Thomas M. Cover,et al.  Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition , 1965, IEEE Trans. Electron. Comput..

[62]  A. Poritz,et al.  Hidden Markov models: a guided tour , 1988, ICASSP-88., International Conference on Acoustics, Speech, and Signal Processing.

[63]  Donald H. Foley Considerations of sample and feature size , 1972, IEEE Trans. Inf. Theory.

[64]  James L. McClelland,et al.  Finite State Automata and Simple Recurrent Networks , 1989, Neural Computation.

[65]  Kenneth W. Bauer,et al.  Integrated feature architecture selection , 1996, IEEE Trans. Neural Networks.

[66]  Harvey F. Silverman,et al.  Hidden Markov model/neural network training techniques for connected alphadigit speech recognition , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[67]  Barak A. Pearlmutter Learning State Space Trajectories in Recurrent Neural Networks , 1989, Neural Computation.

[68]  Thomas Martinetz,et al.  Topology representing networks , 1994, Neural Networks.

[69]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[70]  P. Kahl ISDN implementation strategy of the Deutsche Bundespost telekom , 1990, IEEE Communications Magazine.

[71]  Judith E. Dayhoff,et al.  Adaptive time-delay neural network for temporal correlation and prediction , 1992, Other Conferences.

[72]  David Haussler,et al.  What Size Net Gives Valid Generalization? , 1989, Neural Computation.

[73]  F. Le Chevalier,et al.  Radar target and aspect angle identification , 1978 .

[74]  Esther Levin Hidden control neural architecture modeling of nonlinear time varying systems and its applications , 1993, IEEE Trans. Neural Networks.

[75]  A. Thakoor,et al.  A Decade of Neural Networks: Practical Applications and Prospects , 1994 .

[76]  Mark R. Bower,et al.  Comparison of two neural net classifiers to a quadratic classifier for millimeter-wave radar , 1990, Defense, Security, and Sensing.

[77]  Wright-Patterson Afb,et al.  Feature Selection Using a Multilayer Perceptron , 1990 .

[78]  Stephen Grossberg,et al.  Nonlinear neural networks: Principles, mechanisms, and architectures , 1988, Neural Networks.

[79]  R.P. Lippmann,et al.  Pattern classification using neural networks , 1989, IEEE Communications Magazine.

[80]  P. Lachenbruch An almost unbiased method of obtaining confidence intervals for the probability of misclassification in discriminant analysis. , 1967, Biometrics.

[81]  Robert M. Gray,et al.  An Algorithm for Vector Quantizer Design , 1980, IEEE Trans. Commun..

[82]  Yuichi Nakamura,et al.  Approximation of dynamical systems by continuous time recurrent neural networks , 1993, Neural Networks.

[83]  Barak A. Pearlmutter Dynamic recurrent neural networks , 1990 .

[84]  Geoffrey E. Hinton,et al.  A time-delay neural network architecture for isolated word recognition , 1990, Neural Networks.

[85]  Steven K. Rogers,et al.  Application of a neural architecture to extract motion from image sequences , 1993, Defense, Security, and Sensing.

[86]  F. Murtaghl,et al.  The Multilayer Perceptron for Discriminant Analysis: Two Examples , 1992 .

[87]  James R. Stright Embedded chaotic time series: applications in prediction and spatio-temporal classification , 1994 .

[88]  Steven K. Rogers,et al.  An Approach To Multiple Sensor Target Detection , 1989, Defense, Security, and Sensing.

[89]  Georgia K. Harrup ROC Analysis of IR Segmentation Techniques. , 1994 .

[90]  Steven K. Rogers,et al.  Discrete, spatiotemporal, wavelet multiresolution analysis method for computing optical flow , 1994 .