A permutation test for inference in logistic regression with small‐ and moderate‐sized data sets by D. M. Potter, Statistics in Medicine 2005; 24:693–708

In his very interesting paper, Potter proposes a novel permutation test procedure for inference in the logistic regression model. His so-called permutation of regressor residuals (PRR) test is of particular importance in case of separation, i.e., if the data is such that finite maximum likelihood estimates (MLEs) do not exist. He admits that one alternative applicable whether or not ordinary MLEs exist is a penalized likelihood procedure [1, 2], but erroneously states that inference from this approach had not yet been studied and software was not available. Potter did not mention our paper [3] dealing with this approach in situations of separation. We proposed to perform inference based on penalized likelihood ratio tests and penalized profile likelihood confidence intervals, and studied their small-sample properties in extenso. We also provide software for routine application [4, 5]. We agree with Potter that the incontinence data makes a perfect example for the very situation where an alternative to asymptotic likelihood inference or exact conditional inference is needed. However, it should be mentioned that the penalized likelihood approach provides such an alternative. For the regression parameter of variable x3 of this data set, we calculated a 95% profile penalized likelihood confidence interval of [0.0056, 0.4332] and a penalized likelihood ratio test p-value of 0.0363, which compares well with the PRR test p-value of 0.028.