Robustness of optimal working points for nonadiabatic holonomic quantum computation