Hierarchical multi-class classification in multimodal spacecraft data using DNN and weighted support vector machine

Abstract Prognostics and health management (PHM) is widely applied to assess the reliability, safety and operation of systems particularly in spacecraft systems. However, spacecraft systems are very complex with intangibility and uncertainty, and it is difficult to model and analyze the complex degradation process, and thus there is no single prognostic method for solving the critical and complicated problem. This paper presents a novel hierarchical multi-class classification method using deep neural networks (DNN) and weighted support vector machine (WSVM) in order to achieve a highly discriminative feature representation for classifying the multimodal spacecraft data. First, the stack auto-Encoder (SAE) or deep belief network is adopted to initialize the initial weights and offsets of the hierarchical multi-layer neural network in order to reduce the dimension of the original multimodal data, and the optimal depth of multi-layer neural network and the discriminative features are also obtained. Second, in order to make the high dimensional spacecraft data more separable, the initialization parameters are online monitored by using a gradient descent method. Finally, a flexible hierarchical estimation method of a multi-class weighted support vector machines (MCWSVM) is applied to classify the multimodal spacecraft data. The performance of the proposed work is evaluated by the classification accuracy, sensitivity, specificity and execution time, respectively. The results demonstrate that the proposed DNN with MCWSVM is efficient in terms of better classification accuracy at a lesser execution time when compared to K-nearest neighbors (KNN), SVM and naive Bayes method (NBM).

[1]  Ruifan Li,et al.  Deep correspondence restricted Boltzmann machine for cross-modal retrieval , 2015, Neurocomputing.

[2]  Carmine Clemente,et al.  Robust PCA micro-doppler classification using SVM on embedded systems , 2014, IEEE Transactions on Aerospace and Electronic Systems.

[3]  Zhi-Hua Zhou,et al.  A k-nearest neighbor based algorithm for multi-label classification , 2005, 2005 IEEE International Conference on Granular Computing.

[4]  Michalis Zervakis,et al.  Deep learning for multi-label land cover classification , 2015, SPIE Remote Sensing.

[5]  Heiko Wersing,et al.  Learning Optimized Features for Hierarchical Models of Invariant Object Recognition , 2003, Neural Computation.

[6]  Samy Bengio,et al.  Guest Editors' Introduction: Special Section on Learning Deep Architectures , 2013, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Bin Yu,et al.  Feature learning based on SAE-PCA network for human gesture recognition in RGBD images , 2015, Neurocomputing.

[8]  Ivo Paixao de Medeiros,et al.  Use of PHM Information and System Architecture for Optimized Aircraft Maintenance Planning , 2015, IEEE Systems Journal.

[9]  Zhi-Hua Zhou,et al.  ML-KNN: A lazy learning approach to multi-label learning , 2007, Pattern Recognit..

[10]  Pedro M. Domingos,et al.  On the Optimality of the Simple Bayesian Classifier under Zero-One Loss , 1997, Machine Learning.

[11]  Frank L. Lewis,et al.  Optimal control of nonlinear discrete time-varying systems using a new neural network approximation structure , 2015, Neurocomputing.

[12]  Christian Igel,et al.  Training restricted Boltzmann machines: An introduction , 2014, Pattern Recognit..

[13]  Avinash C. Kak,et al.  PCA versus LDA , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Ana L. N. Fred,et al.  Advances in pattern recognition applications and methods , 2014, Neurocomputing.

[15]  Daoqiang Zhang,et al.  Joint Binary Classifier Learning for ECOC-Based Multi-Class Classification , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Yong Huang,et al.  Multi-parameter decoupling and slope tracking control strategy of a large-scale high altitude environment simulation test cabin , 2014 .

[17]  Yong Huang,et al.  An intelligent control method for a large multi-parameter environmental simulation cabin , 2013 .

[18]  Kunikazu Kobayashi,et al.  Time series forecasting using a deep belief network with restricted Boltzmann machines , 2014, Neurocomputing.

[19]  Gang Wang,et al.  Mapping the effect of escitalopram treatment on amplitude of low-frequency fluctuations in patients with depression: a resting-state fMRI study , 2017, Metabolic Brain Disease.

[20]  K.C. Chang,et al.  K-nearest neighbor particle filters for dynamic hybrid Bayesian networks , 2008, IEEE Transactions on Aerospace and Electronic Systems.

[21]  Daoqiang Zhang,et al.  Relationship Induced Multi-Template Learning for Diagnosis of Alzheimer’s Disease and Mild Cognitive Impairment , 2016, IEEE Transactions on Medical Imaging.

[22]  Irina Rish,et al.  An empirical study of the naive Bayes classifier , 2001 .

[23]  Xueming Qian,et al.  Image classification based on low-rank matrix recovery and Naive Bayes collaborative representation , 2015, Neurocomputing.

[24]  Geoffrey E. Hinton,et al.  Application of Deep Belief Networks for Natural Language Understanding , 2014, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[25]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[26]  Ana L. N. Fred,et al.  Advances in pattern recognition applications and methods , 2016, Neurocomputing.

[27]  Shaohong Li,et al.  One-Dimensional Frequency-Domain Features for Aircraft Recognition from Radar Range Profiles , 2010, IEEE Transactions on Aerospace and Electronic Systems.

[28]  Yang Li,et al.  A Spacecraft Electrical Characteristics Multi-Label Classification Method Based on Off-Line FCM Clustering and On-Line WPSVM , 2015, PloS one.

[29]  Yingmin Jia,et al.  Neural network-based distributed adaptive attitude synchronization control of spacecraft formation under modified fast terminal sliding mode , 2016, Neurocomputing.

[30]  Daoqiang Zhang,et al.  Two-Stage Cost-Sensitive Learning for Software Defect Prediction , 2014, IEEE Transactions on Reliability.

[31]  Daoqiang Zhang,et al.  Pairwise Constraint-Guided Sparse Learning for Feature Selection , 2016, IEEE Transactions on Cybernetics.

[32]  Ujjwal Maulik,et al.  A parallel bi-directional self-organizing neural network (PBDSONN) architecture for color image extraction and segmentation , 2012, Neurocomputing.

[33]  Chen Jing,et al.  SVM and PCA based fault classification approaches for complicated industrial process , 2015, Neurocomputing.

[34]  Yue Gao,et al.  3-D Object Retrieval and Recognition With Hypergraph Analysis , 2012, IEEE Transactions on Image Processing.

[35]  Ju-Jang Lee,et al.  Adaptive control for uncertain nonlinear systems based on multiple neural networks , 2004, IEEE Trans. Syst. Man Cybern. Part B.

[36]  T.D. Batzel,et al.  Prognostic Health Management of Aircraft Power Generators , 2009, IEEE Transactions on Aerospace and Electronic Systems.

[37]  Brian P. Salmon,et al.  Multiview Deep Learning for Land-Use Classification , 2015, IEEE Geoscience and Remote Sensing Letters.

[38]  Jun Zhang,et al.  Local Energy Pattern for Texture Classification Using Self-Adaptive Quantization Thresholds , 2013, IEEE Transactions on Image Processing.

[39]  Long Zhang,et al.  Material identification of loose particles in sealed electronic devices using PCA and SVM , 2015, Neurocomputing.

[40]  Jiuping Xu,et al.  PHM-Oriented Integrated Fusion Prognostics for Aircraft Engines Based on Sensor Data , 2014, IEEE Sensors Journal.

[41]  Dinggang Shen,et al.  Sparse Multivariate Autoregressive Modeling for Mild Cognitive Impairment Classification , 2014, Neuroinformatics.

[42]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.