A Recurrent Probabilistic Neural Network with Dimensionality Reduction Based on Time-series Discriminant Component Analysis

This paper proposes a probabilistic neural network (NN) developed on the basis of time-series discriminant component analysis (TSDCA) that can be used to classify high-dimensional time-series patterns. TSDCA involves the compression of high-dimensional time series into a lower dimensional space using a set of orthogonal transformations and the calculation of posterior probabilities based on a continuous-density hidden Markov model with a Gaussian mixture model expressed in the reduced-dimensional space. The analysis can be incorporated into an NN, which is named a time-series discriminant component network (TSDCN), so that parameters of dimensionality reduction and classification can be obtained simultaneously as network coefficients according to a backpropagation through time-based learning algorithm with the Lagrange multiplier method. The TSDCN is considered to enable high-accuracy classification of high-dimensional time-series patterns and to reduce the computation time taken for network training. The validity of the TSDCN is demonstrated for high-dimensional artificial data and electroencephalogram signals in the experiments conducted during the study.

[1]  Jian Pei,et al.  A brief survey on sequence classification , 2010, SKDD.

[2]  Sanjoy Dasgupta,et al.  Adaptive Control Processes , 2010, Encyclopedia of Machine Learning and Data Mining.

[3]  Toshio Tsuji,et al.  A recurrent log-linearized Gaussian mixture network , 2003, IEEE Trans. Neural Networks.

[4]  Roy L. Streit,et al.  Maximum likelihood training of probabilistic neural networks , 1994, IEEE Trans. Neural Networks.

[5]  Lars Schmidt-Thieme,et al.  INSIGHT: Efficient and Effective Instance Selection for Time-Series Classification , 2011, PAKDD.

[6]  Geoffrey E. Hinton,et al.  Phoneme recognition using time-delay neural networks , 1989, IEEE Trans. Acoust. Speech Signal Process..

[7]  Daniel Lemire,et al.  Time series classification by class-specific Mahalanobis distance measures , 2010, Adv. Data Anal. Classif..

[8]  Terry Caelli,et al.  Model-based neural networks , 1993, Neural Networks.

[9]  P J Webros.  BACKPROPAGATION THROUGH TIME: WHAT IT DOES AND HOW TO DO IT , 1990 .

[10]  Ling Guan,et al.  Modularity in neural computing , 1999, Proc. IEEE.

[11]  Abdulhamit Subasi,et al.  EEG signal classification using PCA, ICA, LDA and support vector machines , 2010, Expert Syst. Appl..

[12]  S. Holm.  A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[13]  Donald F. Specht,et al.  A general regression neural network , 1991, IEEE Trans. Neural Networks.

[14]  R. Bellman,et al.  V. Adaptive Control Processes , 1964 .

[15]  Michael I. Jordan,et al.  Distance Metric Learning with Application to Clustering with Side-Information , 2002, NIPS.

[16]  J. G. Snodgrass,et al.  A standardized set of 260 pictures: norms for name agreement, image agreement, familiarity, and visual complexity. , 1980, Journal of experimental psychology. Human learning and memory.

[17]  Xuelong Li,et al.  General Tensor Discriminant Analysis and Gabor Features for Gait Recognition , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[19]  Sangita M. Rajput,et al.  Classification of EEG using PCA, ICA and Neural Network , 2012 .

[20]  E. M. Wright,et al.  Adaptive Control Processes: A Guided Tour , 1961, The Mathematical Gazette.

[21]  Sabri Koçer,et al.  Classification of EMG Signals Using PCA and FFT , 2005, Journal of Medical Systems.

[22]  Dong Xu,et al.  Video Event Recognition Using Kernel Methods with Multilevel Temporal Alignment , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Sahin Albayrak,et al.  Pattern recognition and classification for multivariate time series , 2011, SensorKDD '11.

[24]  Alexandros Nanopoulos,et al.  Time-Series Classification in Many Intrinsic Dimensions , 2010, SDM.

[25]  Xuelong Li,et al.  Geometric Mean for Subspace Selection , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Michael I. Jordan.  Serial Order: A Parallel Distributed Processing Approach , 1997 .

[27]  Jianting Cao,et al.  CLASSIFICATION OF SINGLE TRIAL EEG SIGNALS BY A COMBINED PRINCIPAL + INDEPENDENT COMPONENT ANALYSIS AND PROBABILISTIC NEURAL NETWORK APPROACH , 2003 .

[28]  Dacheng Tao,et al.  Double Shrinking Sparse Dimension Reduction , 2013, IEEE Transactions on Image Processing.

[29]  Michael S. Lazar,et al.  Spatial patterns underlying population differences in the background EEG , 2005, Brain Topography.

[30]  Jason Lines,et al.  A shapelet transform for time series classification , 2012, KDD.

[31]  Feiping Nie,et al.  Extracting the optimal dimensionality for local tensor discriminant analysis , 2009, Pattern Recognit..

[32]  Feiping Nie,et al.  Learning a Mahalanobis distance metric for data clustering and classification , 2008, Pattern Recognit..

[33]  G. Pfurtscheller,et al.  Designing optimal spatial filters for single-trial EEG classification in a movement task , 1999, Clinical Neurophysiology.

[34]  Ravi Kothari,et al.  Bayes-optimality motivated linear and multilayered perceptron-based dimensionality reduction , 2000, IEEE Trans. Neural Networks Learn. Syst..

[35]  Ivor W. Tsang,et al.  Flexible Manifold Embedding: A Framework for Semi-Supervised and Unsupervised Dimension Reduction , 2010, IEEE Transactions on Image Processing.

[36]  Dong Xu,et al.  Semi-Supervised Dimension Reduction Using Trace Ratio Criterion , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[37]  Toshio Tsuji,et al.  EEG discrimination using wavelet packet transform and a reduced-dimensional recurrent neural network , 2010, Proceedings of the 10th IEEE International Conference on Information Technology and Applications in Biomedicine.

[38]  Manish Kumar,et al.  Face Recognition Using Principle Component Analysis, Eigenface and Neural Network , 2010, 2010 International Conference on Signal Acquisition and Processing.

[39]  Toshio Tsuji,et al.  Bioelectric signal classification using a recurrent probabilistic neural network with time-series discriminant component analysis , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[40]  Feiping Nie,et al.  Neighborhood MinMax Projections , 2007, IJCAI.

[41]  A. Asuncion,et al.  UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences , 2007 .

[42]  Leonid I. Perlovsky,et al.  Maximum likelihood neural networks for sensor fusion and adaptive classification , 1991, Neural Networks.

[43]  Xuelong Li,et al.  General Averaged Divergence Analysis , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[44]  Michail Zak,et al.  Terminal attractors in neural networks , 1989, Neural Networks.

[45]  Olufemi A. Omitaomu,et al.  Weighted dynamic time warping for time series classification , 2011, Pattern Recognit..