Robust Adaptation of Foundation Models with Black-Box Visual Prompting