The Subjective Nature of Outlier Rejection Procedures

SUMMARY Procedures for rejecting outliers are essentially two stage, involving first an individual's judgment that a value in a given set of data is surprising, and then testing the surprising value for discordancy. The conventional frequency interpretation of the significance level for outlier tests is shown to be invalid and an experiment designed to investigate the inherent subjective nature of outlier procedures is described. This confirms that there are variations both between individuals in their reaction to surprising values and also between judgments made by the same individual on different occasions. The method of presentation of the data is shown to affect an individual's ability to perceive possible outliers. More surprising, it turns out that factors such as scale and pattern of the data are also very relevant.