Robustness of inflation to large tensor perturbations