A Hierarchical Neural Network Architecture for Classification

In this paper, a hierarchical neural network with cascading architecture is proposed and its application to classification is analyzed. This cascading architecture consists of multiple levels of neural network structure, in which the outputs of the hidden neurons in the higher hierarchical level are treated as an equivalent input data to the input neurons at the lower hierarchical level. The final predictive result is obtained through a modified weighted majority vote scheme. In this way, it is hoped that new patterns could be learned from hidden layers at each level and thus the combination result could significantly improve the learning performance of the whole system. In simulation, a comparison experiment is carried out among our approach and two popular ensemble learning approaches, bagging and AdaBoost. Various simulation results based on synthetic data and real data demonstrate this approach can improve the classification performance.

[1]  L. Breiman.  Arcing classifier (with discussion and a rejoinder by the author) , 1998 .

[2]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[3]  Haibo He,et al.  Learning from Imbalanced Data , 2009, IEEE Transactions on Knowledge and Data Engineering.

[4]  Honglak Lee,et al.  Unsupervised learning of hierarchical representations with convolutional deep belief networks , 2011, Commun. ACM.

[5]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[6]  Ruoyu Li,et al.  Data Mining Based Full Ceramic Bearing Fault Diagnostic System Using AE Sensors , 2011, IEEE Transactions on Neural Networks.

[7]  Aiguo Li,et al.  Fraud Detection in Tax Declaration Using Ensemble ISGNN , 2009, 2009 WRI World Congress on Computer Science and Information Engineering.

[8]  Haibo He,et al.  IMORL: Incremental Multiple-Object Recognition and Localization , 2008, IEEE Transactions on Neural Networks.

[9]  R. Polikar,et al.  Bootstrap - Inspired Techniques in Computation Intelligence , 2007, IEEE Signal Processing Magazine.

[10]  R. Polikar,et al.  Ensemble based systems in decision making , 2006, IEEE Circuits and Systems Magazine.

[11]  Chun-Jung Chen,et al.  Power System Stabilizer Using a New Recurrent Neural Network for Multi-Machine , 2006, 2006 IEEE International Power and Energy Conference.

[12]  Wei-Yang Lin,et al.  Machine Learning in Financial Crisis Prediction: A Survey , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[13]  Byoung-Tak Zhang,et al.  Ensemble Learning with Active Example Selection for Imbalanced Biomedical Data Classification , 2011, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[14]  Dimitris N. Metaxas,et al.  Automated detection of prostatic adenocarcinoma from high-resolution ex vivo MRI , 2005, IEEE Transactions on Medical Imaging.

[15]  Haibo He,et al.  Incremental Learning From Stream Data , 2011, IEEE Transactions on Neural Networks.

[16]  Haibo He,et al.  Ensemble learning for wind profile prediction with missing values , 2011, Neural Computing and Applications.

[17]  Klaus-Robert Müller,et al.  Efficient BackProp , 2012, Neural Networks: Tricks of the Trade.

[18]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[19]  Grgoire Montavon,et al.  Neural Networks: Tricks of the Trade , 2012, Lecture Notes in Computer Science.

[20]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.