Logistic Regression with Robust Bootstrapping

The bootstrap method is a widely used technique for statistical learning and inference. However, its performance can be dramatically affected by outliers in the training data. In this paper, we propose two robust bootstrapping algorithms for logistic regression. These methods have several compelling advantages: they alleviate the dependency on fine-tuning the hyperparameters, and can still remain robust against highly contaminated data. The performance of the proposed algorithms is evaluated on synthetic and real-world data sets.

[1]  Samuel Müller,et al.  Outlier Robust Model Selection in Linear Regression , 2005 .

[2]  Lawrence Carin,et al.  Sparse multinomial logistic regression: fast algorithms and generalization bounds , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Kaushik K. Majumdar,et al.  Single-Trial EEG Classification Using Logistic Regression Based on Ensemble Synchronization , 2014, IEEE Journal of Biomedical and Health Informatics.

[4]  Dalton S. Rosario.  Highly effective logistic regression model for signal (anomaly) detection , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[5]  Lyle H. Ungar,et al.  Machine Learning manuscript No. (will be inserted by the editor) Active Learning for Logistic Regression: , 2007 .

[6]  Ana M. Pires,et al.  ROBUST BOOTSTRAP : AN ALTERNATIVE TO BOOTSTRAPPING ROBUST ESTIMATORS , 2014 .

[7]  David Hinkley,et al.  Bootstrap Methods: Another Look at the Jackknife , 2008 .

[8]  J. Habbema,et al.  Prognostic modelling with logistic regression analysis: a comparison of selection and estimation methods in small data sets. , 2000, Statistics in medicine.

[9]  Michael Muma,et al.  Robust bootstrap methods with an application to geolocation in harsh LOS/NLOS environments , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[10]  Conceição Amado,et al.  Robust Bootstrap with Non Random Weights Based on the Influence Function , 2004 .

[11]  Shie Mannor,et al.  Robust Logistic Regression and Classification , 2014, NIPS.

[12]  F. Pampel.  Logistic Regression: A Primer , 2000 .

[13]  Gutti Jogesh Babu.  Bootstrap Techniques for Signal Processing , 2005, Technometrics.

[14]  Heikki Huttunen,et al.  Image segmentation using sparse logistic regression with spatial prior , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[15]  Frederick R. Forst,et al.  On robust estimation of the location parameter , 1980 .