Guaranteed Robustness with Fast Adaptation