Cost-Sensitive Convolution based Neural Networks for Imbalanced Time-Series Classification

Some deep convolutional neural networks were proposed for time-series classification and class imbalanced problems. However, those models performed degraded and even failed to recognize the minority class of an imbalanced temporal sequences dataset. Minority samples would bring troubles for temporal deep learning classifiers due to the equal treatments of majority and minority class. Until recently, there were few works applying deep learning on imbalanced time-series classification (ITSC) tasks. Here, this paper aimed at tackling ITSC problems with deep learning. An adaptive cost-sensitive learning strategy was proposed to modify temporal deep learning models. Through the proposed strategy, classifiers could automatically assign misclassification penalties to each class. In the experimental section, the proposed method was utilized to modify five neural networks. They were evaluated on a large volume, real-life and imbalanced time-series dataset with six metrics. Each single network was also tested alone and combined with several mainstream data samplers. Experimental results illustrated that the proposed cost-sensitive modified networks worked well on ITSC tasks. Compared to other methods, the cost-sensitive convolution neural network and residual network won out in the terms of all metrics. Consequently, the proposed cost-sensitive learning strategy can be used to modify deep learning classifiers from cost-insensitive to cost-sensitive. Those cost-sensitive convolutional networks can be effectively applied to address ITSC issues.

[1]  Mohammed Bennamoun,et al.  Cost-Sensitive Learning of Deep Feature Representations From Imbalanced Data , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[2]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[3]  Chengqi Zhang,et al.  A Comparative Study of Sampling Methods and Algorithms for Imbalanced Time Series Classification , 2012, Australasian Conference on Artificial Intelligence.

[4]  Yixin Chen,et al.  Multi-Scale Convolutional Neural Networks for Time Series Classification , 2016, ArXiv.

[5]  Marcin Michalak,et al.  Predicting seismic events in coal mines based on underground sensor measurements , 2017, Eng. Appl. Artif. Intell..

[6]  Ashfaqur Rahman,et al.  Convolutional Neural Network for Time Series Cattle Behaviour Classification , 2016, TSAA '16.

[7]  Atsuto Maki,et al.  A systematic study of the class imbalance problem in convolutional neural networks , 2017, Neural Networks.

[8]  Li Wei,et al.  Fast time series classification using numerosity reduction , 2006, ICML.

[9]  Tiejun Huang,et al.  CNUSVM: Hybrid CNN-Uneven SVM Model for Imbalanced Visual Learning , 2016, 2016 IEEE Second International Conference on Multimedia Big Data (BigMM).

[10]  Longbing Cao,et al.  Training deep neural networks on imbalanced data sets , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[11]  Tim Oates,et al.  Time series classification from scratch with deep neural networks: A strong baseline , 2016, 2017 International Joint Conference on Neural Networks (IJCNN).

[12]  Houshang Darabi,et al.  LSTM Fully Convolutional Networks for Time Series Classification , 2017, IEEE Access.

[13]  Xindong Wu,et al.  10 Challenging Problems in Data Mining Research , 2006, Int. J. Inf. Technol. Decis. Mak..

[14]  Chen Huang,et al.  Learning Deep Representation for Imbalanced Classification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Haishuai Wang,et al.  Cost-sensitive Deep Learning for Early Readmission Prediction at A Major Hospital , 2017 .

[16]  Igor Kononenko,et al.  Cost-Sensitive Learning with Neural Networks , 1998, ECAI.

[17]  Zhi-Hua Zhou,et al.  Ieee Transactions on Knowledge and Data Engineering 1 Training Cost-sensitive Neural Networks with Methods Addressing the Class Imbalance Problem , 2022 .

[18]  Francisco Herrera,et al.  An insight into classification with imbalanced data: Empirical results and current trends on using data intrinsic characteristics , 2013, Inf. Sci..

[19]  Longin Jan Latecki,et al.  Improving SVM classification on imbalanced time series data sets with ghost points , 2011, Knowledge and Information Systems.

[20]  Fernando Nogueira,et al.  Imbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine Learning , 2016, J. Mach. Learn. Res..

[21]  Joelle Pineau,et al.  Learning Robust Features using Deep Learning for Automatic Seizure Detection , 2016, MLHC.

[22]  Lawrence Carin,et al.  Earliness-Aware Deep Convolutional Networks for Early Time Series Classification , 2016, ArXiv.

[23]  Min Chen,et al.  Deep Learning for Imbalanced Multimedia Data Classification , 2015, 2015 IEEE International Symposium on Multimedia (ISM).

[24]  Yijing Li,et al.  Learning from class-imbalanced data: Review of methods and applications , 2017, Expert Syst. Appl..

[25]  Nikou Günnemann,et al.  Predicting Defective Engines using Convolutional Neural Networks on Temporal Vibration Signals , 2017, LIDTA@PKDD/ECML.

[26]  Yi Zheng,et al.  Time Series Classification Using Multi-Channels Deep Convolutional Neural Networks , 2014, WAIM.

[27]  U. Rajendra Acharya,et al.  A deep convolutional neural network model to classify heartbeats , 2017, Comput. Biol. Medicine.

[28]  Yi Zheng,et al.  Exploiting multi-channels deep convolutional neural networks for multivariate time series classification , 2015, Frontiers of Computer Science.

[29]  Min Chen,et al.  Deep Learning with MCA-based Instance Selection and Bootstrapping for Imbalanced Data Classification , 2015, 2015 IEEE Conference on Collaboration and Internet Computing (CIC).

[30]  Enhong Chen,et al.  Exploiting MultiChannels Deep Convolutional Neural Networks for Multivariate Time Series Classification , 2015 .

[31]  See-Kiong Ng,et al.  Integrated Oversampling for Imbalanced Time Series Classification , 2013, IEEE Transactions on Knowledge and Data Engineering.

[32]  Stanislas Chambon,et al.  A Deep Learning Architecture for Temporal Sleep Stage Classification Using Multivariate and Multimodal Time Series , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[33]  Yong Duan,et al.  Early classification on multivariate time series , 2015, Neurocomputing.

[34]  Jian Wang,et al.  Fault Detection for the class Imbalance Problem in semiconductor manufacturing Processes , 2014, J. Circuits Syst. Comput..

[35]  Seetha Hari,et al.  Learning From Imbalanced Data , 2019, Advances in Computer and Electrical Engineering.

[36]  Andrew K. C. Wong,et al.  A Weight-Selection Strategy on Training Deep Neural Networks for Imbalanced Classification , 2017, ICIAR.

[37]  Guohua Liang,et al.  An Effective Method for Imbalanced Time Series Classification: Hybrid Sampling , 2013, Australasian Conference on Artificial Intelligence.

[38]  Stefan Wermter,et al.  Towards Effective Classification of Imbalanced Data with Convolutional Neural Networks , 2016, ANNPR.

[39]  John Cristian Borges Gamboa,et al.  Deep Learning for Time-Series Analysis , 2017, ArXiv.