How to deal with "The language-as-fixed-effect fallacy": Common misconceptions and alternative solutions.

Although Clark's (1973) critique of statistical procedures in language and memory studies (the “language-as-fixed-effect fallacy”) has had a profound effect on the way such analyses have been carried out in the past 20 years, it seems that the exact nature of the problem and the proposed solution have not been understood very well. Many investigators seem to assume that generalization to both the subject population and the language as a whole is automatically ensured if separate subject (F1) and item (F2) analyses are performed and that the null hypothesis may safely be rejected if these F values are both significant. Such a procedure is, however, unfounded and not in accordance with the recommendations of Clark (1973). More importantly and contrary to current practice, in many cases there is no need to perform separate subject and item analyses since the traditional F1 is the correct test statistic. In particular this is the case when item variability is experimentally controlled by matching or by counterbalancing.