Statistical Significance, Effect Size Reporting, and Confidence Intervals: Best Reporting Strategies.

With great interest I read the May 2002 editorial in the Journal for Research in Mathematics Education (JRME) (King, 2002) regarding changes to the 5th edition of the Publication Manual of the American Psychological Association (APA, 2001). Of special note to me, and of great import to the field of mathematics education research, are changes regarding the reporting of results from statistical analyses. In particular, researchers are urged to use effect size and confidence interval (CI) reporting in conjunction with null hypothesis statistical significance testing (NHST). The APA Publication Manual now states that for a reader to understand the importance of a reported p-value ". .. it is almost always necessary to include some index of effect size" (APA, 2001, p. 25). The manual further advises: "Because confidence intervals combine information on location and precision and can often be directly used to infer significance levels, they are, in general, the best reporting strategy" (APA, 2001, p. 22). Because this kind of statistical reporting has not been commonplace in mathematics education research and because a spirited exchange of views on methodological issues can facilitate progress in the field, I offer here some discussion of NHST, effect size, and CI reporting to characterize some of the potential contributions of these latter methods to educational research.

[1]  Leland Wilkinson,et al.  Statistical Methods in Psychology Journals Guidelines and Explanations , 2005 .

[2]  Robert M. Capraro,et al.  Treatments of Effect Sizes and Statistical Significance Tests in Textbooks , 2002 .

[3]  Olejnik,et al.  Measures of Effect Size for Comparative Studies: Applications, Interpretations, and Limitations. , 2000, Contemporary educational psychology.

[4]  Elazar J. Pedhazur,et al.  Measurement, Design, and Analysis: An Integrated Approach , 1994 .

[5]  Larry G. Daniel.  The Statistical Significance Controversy Is Definitely Not Over: A Rejoinder to Responses by Thompson, Knapp, and Levin. , 1998 .

[6]  J. Kilpatrick.  Where's the Evidence? , 2001 .

[7]  Roger E. Kirk,et al.  Promoting Good Statistical Practices: Some Suggestions , 2001 .

[8]  Robert M. Capraro,et al.  Measurement Error of Scores on the Mathematics Anxiety Rating Scale across Studies , 2001 .

[9]  Jacob Cohen.  The earth is round (p < .05) , 1994 .

[10]  R. Nickerson,et al.  Null hypothesis significance testing: a review of an old and continuing controversy. , 2000, Psychological methods.

[11]  Neil Thomason,et al.  Reporting of statistical inference in the Journal of Applied Psychology : Little evidence of reform. , 2001 .

[12]  James E. McLean,et al.  Fight the Good Fight: A Response to Thompson, Knapp, and Levin , 1998 .

[13]  Learning About What Fits: Preschool Children's Reasoning About Effects of Object Size , 2002 .

[14]  R. Kirk.  Practical Significance: A Concept Whose Time Has Come , 1996 .

[15]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[16]  James Friedrich,et al.  Statistical Training in Psychology: A National Survey and Commentary on Undergraduate Programs , 2000 .

[17]  Xitao Fan,et al.  Statistical Significance and Effect Size in Education Research: Two Sides of a Coin , 2001 .

[18]  Bruce Thompson,et al.  "Statistical," "practical", and "clinical": How many kinds of significance do counselors need to consider? , 2002 .