Non-perturbative effects in spin glasses