论文信息 - Deep Incremental Boosting

Deep Incremental Boosting

This paper introduces Deep Incremental Boosting, a new technique derived from AdaBoost, specifically adapted to work with Deep Learning methods, that reduces the required training time and improves generalisation. We draw inspiration from Transfer of Learning approaches to reduce the start-up time to training each incremental Ensemble member. We show a set of experiments that outlines some preliminary results on some common Deep Learning datasets and discuss the potential improvements Deep Incremental Boosting brings to traditional Ensemble methods in Deep Learning.

George D. Magoulas | Alan Mosca | G. D. Magoulas | Alan Mosca

[1] Yann LeCun,et al. The mnist database of handwritten digits , 2005 .

[2] Qiang Chen,et al. Network In Network , 2013, ICLR.

[3] Yoav Freund,et al. Experiments with a New Boosting Algorithm , 1996, ICML.

[4] R. Schapire. The Strength of Weak Learnability , 1990, Machine Learning.

[5] Thomas Brox,et al. Striving for Simplicity: The All Convolutional Net , 2014, ICLR.

[6] Yann LeCun,et al. Regularization of Neural Networks using DropConnect , 2013, ICML.

[7] Jimmy Ba,et al. Adam: A Method for Stochastic Optimization , 2014, ICLR.

[8] Alex Krizhevsky,et al. Learning Multiple Layers of Features from Tiny Images , 2009 .

[9] Yoshua Bengio,et al. How transferable are features in deep neural networks? , 2014, NIPS.

[10] Yoshua Bengio,et al. Deep Learning of Representations for Unsupervised and Transfer Learning , 2011, ICML Unsupervised and Transfer Learning.

[11] Thomas G. Dietterich. An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization , 2000, Machine Learning.

[12] Benjamin Graham,et al. Fractional Max-Pooling , 2014, ArXiv.

[13] Rich Caruana,et al. Do Deep Nets Really Need to be Deep? , 2013, NIPS.