Methods of Handling Sequence Effects in Human Factors Engineering Experiments.

Abstract : A common procedure in human factors engineering experiments is to test the same subject sequentially on a series of different experimental conditions. These 'within subjects' designs produce sequence effects that may be unwanted (in comparing equipment designs) or of considerable interest (in developing training devices). This report identifies the more common sequence effects and describes a variety of procedures, experimental designs, and statistical techniques to minimize, adjust for, or isolate these effects. Change-over designs are described that permit reasonably unbiased estimates of treatment effects when it is suspected that residual effects from prior treatments may have carried over to affect the direct performance at the time of measurement. Designs are described that enable reasonably unbiased estimates of treatment effects to be made even in the presence of underlying shifts in performance when such trend effects cannot be associated with changes in experimental conditions. Other designs recommend sequences that may be robust against trend effects while limiting the number of times the level of one or more factor must be changed as experimental conditions are changed. Some methods of comparing the effectiveness of different treatment presentation orders and for optimizing training schedules are also discussed.