Improved Worst-Group Robustness via Classifier Retraining on Independent Splits

High-capacity deep neural networks (DNNs) trained with Empirical Risk Minimization (ERM) often suffer from poor worst-group accuracy despite good on-average performance, where worst-group accuracy measures a model’s robustness towards certain subpopulations of the input space. Spurious correlations and memorization behaviors of ERM trained DNNs are typically attributed to this degradation in performance. We develop a method, called CRIS, that address these issues by performing robust classifier retraining on independent splits of the dataset. This results in a simple method that improves upon state-of-the-art methods, such as Group DRO, on standard datasets while relying on much fewer group labels and little additional hyperparameter tuning. validation accuracies on the feature extractor’s quality (measured by robust performance after classifier retraining) is presented in Appendix B.2. The data show that there is a positive correlation between validation average accuracy and the model’s features’ quality. This observation serves as a proxy for CRIS’s model selection criterion. A similar positive correlation is observed when validation worst-group accuracy is used.