Improving Accuracy by Rejecting Outliers?

Inspecting quality into batches It may be helpful to begin by describing a related and very easily understood phenomenon in the area of quality control. Suppose we have very large batches of widgets, and every batch contains exactly 5% of defective items. Not happy with this, we set up an inspection system, sampling 100 widgets from each batch and rejecting every batch with more than four defectives in the sample of 100. If the samples are random, the number of defectives in the sample has a binomial distribution with parameters N = 100 and p = 0.05. Some calculations with this distribution show that in the long run we will reject 56% of the batches. however, all the accepted batches, not to mention all the rejected ones, still contain exactly 5% of defective items and we have not improved the quality of the product at all. Sampling inspection is good for detecting the occasional bad batch and thus flagging a problem with production. It is no good at all for improving the quality of batches when the quality is completely uniform, and very little use when the quality varies only a little.