Fault diagnosis based on pruned ensemble

A new fault diagnosis method based on ensemble pruning is proposed. Ensemble pruning means to search for a good subset of ensemble members that performs as well as, or better than, the original ensemble. Margin distribution on training sets is thought as an important factor to improve the generalization performance of classifiers. In this paper, based on the margin loss minimization, a new ensemble pruning algorithm is proposed and utilized in fault diagnosis. Experiment results show the effectiveness of the proposed technique.

[1]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[2]  John Shawe-Taylor,et al.  A framework for structural risk minimisation , 1996, COLT '96.

[3]  Peter L. Bartlett,et al.  For Valid Generalization the Size of the Weights is More Important than the Size of the Network , 1996, NIPS.

[4]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[5]  Yoav Freund,et al.  Boosting the margin: A new explanation for the effectiveness of voting methods , 1997, ICML.

[6]  Peter L. Bartlett,et al.  The Sample Complexity of Pattern Classification with Neural Networks: The Size of the Weights is More Important than the Size of the Network , 1998, IEEE Trans. Inf. Theory.

[7]  Trevor Hastie,et al.  Additive Logistic Regression : a Statistical , 1998 .

[8]  John Shawe-Taylor,et al.  Boosting the Margin Distribution , 2000, IDEAL.

[9]  J. Friedman.  Special Invited Paper-Additive logistic regression: A statistical view of boosting , 2000 .

[10]  Osamu Watanabe,et al.  MadaBoost: A Modification of AdaBoost , 2000, COLT.

[11]  John Shawe-Taylor,et al.  Generalisation Error Bounds for Sparse Linear Classifiers , 2000, COLT.

[12]  Wei Tang,et al.  Ensembling neural networks: Many could be better than all , 2002, Artif. Intell..

[13]  Tom Heskes,et al.  Clustering ensembles of neural network models , 2003, Neural Networks.

[14]  Sally Floyd,et al.  Sample compression, learnability, and the Vapnik-Chervonenkis dimension , 2004, Machine Learning.

[15]  Naftali Tishby,et al.  Margin based feature selection - theory and algorithms , 2004, ICML.

[16]  Ho-Chan Kim,et al.  A real-time intelligent multiple fault diagnostic system , 2006 .

[17]  William Nick Street,et al.  Ensemble Pruning Via Semi-definite Programming , 2006, J. Mach. Learn. Res..

[18]  Xin Yao,et al.  An analysis of diversity measures , 2006, Machine Learning.

[19]  B. Lee,et al.  A real-time intelligent multiple fault diagnostic system , 2006 .

[20]  Daniel Hernández-Lobato,et al.  Pruning Adaptive Boosting Ensembles by Means of a Genetic Algorithm , 2006, IDEAL.

[21]  Gonzalo Martínez-Muñoz,et al.  Pruning in ordered bagging ensembles , 2006, ICML.

[22]  Gonzalo Martínez-Muñoz,et al.  Using boosting to prune bagging ensembles , 2007, Pattern Recognit. Lett..

[23]  Kihoon Choi,et al.  Application of an Effective Data-Driven Approach to Real-time time Fault Diagnosis in Automotive Engines , 2007, 2007 IEEE Aerospace Conference.

[24]  Zhi-Hua Zhou,et al.  On the Margin Explanation of Boosting Algorithms , 2008, COLT.

[25]  Wei Tian.  A novel intelligent fault diagnosis method using entropy-based rough sets and its application , 2008, 2008 7th World Congress on Intelligent Control and Automation.

[26]  Huanhuan Chen,et al.  Predictive Ensemble Pruning by Expectation Propagation , 2009, IEEE Transactions on Knowledge and Data Engineering.

[27]  Daniel Hernández-Lobato,et al.  An Analysis of Ensemble Pruning Techniques Based on Ordered Aggregation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Chunhua Shen,et al.  Boosting Through Optimization of Margin Distributions , 2009, IEEE Transactions on Neural Networks.

[29]  Li Zhang,et al.  On the sparseness of 1-norm support vector machines , 2010, Neural Networks.

[30]  Li Zhang,et al.  Sparse ensembles using weighted combination methods based on linear programming , 2011, Pattern Recognit..