Analysis of the Astronomy Diagnostic Test

Seventy undergraduate class sections were examined from the database of Astronomy Diagnostic Test (ADT) results of Deming and Hufnagel to determine if course format correlated with ADT normalized gain scores. Normalized gains were calculated for four different classroom scenarios: lecture, lecture with discussion, lecture with lab, and lecture with both lab and discussion. Statistical analysis shows that there are no significant differences in normalized gain among the self- reported classroom formats. Prerequisites related to mathematics courses did show differences in normalized gain. Of all reported course activities, only the lecture and the readings for the course correlate significantly with the normalized gain. This analysis suggests that the ADT may not have enough sensitivity to measure differences in the effectiveness of different course formats because of the wide range of topics that the ADT addresses with few questions. Different measures of gain and their biases are discussed. We argue that the use of the normalized gain is not always warranted because of its strong bias toward high pretest scores.

[1]  B. Hufnagel,et al.  Development of the Astronomy Diagnostic Test , 2001 .

[2]  Grace L. Deming,et al.  Who's taking ASTRO 101? , 2001 .

[3]  M. Zeilik,et al.  Birth of the Astronomy Diagnostic Test: Prototest Evolution , 2001 .

[4]  Lei Bao,et al.  Theoretical comparisons of average normalized gain calculations , 2006 .

[5]  Janelle M. Bailey,et al.  Development Of A Concept Inventory To Assess Students’ Understanding And Reasoning Difficulties About The Properties And Formation Of Stars , 2006 .

[6]  Timothy F. Slater,et al.  Development and Validation of the Light and Spectroscopy Concept Inventory , 2006 .

[7]  R. Hake.  Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses , 1998 .

[8]  Grace L. Deming.  Results from the Astronomy Diagnostic Test National Project , 2001 .

[9]  Sidney C. Wolff,et al.  Astronomy Education Review 1 , 2006 .

[10]  D. Hestenes,et al.  Force concept inventory , 1992 .

[11]  B. Thompson.  Effect sizes, confidence intervals, and confidence intervals for effect sizes , 2007 .

[12]  Rebecca Susan Lindell.  Enhancing college students' understanding of lunar phases , 2001 .

[13]  Edward L. Deci,et al.  Intrinsic Motivation and Self-Determination in Human Behavior , 1975, Perspectives in Social Psychology.

[14]  Grace L. Deming,et al.  The Astronomy Diagnostic Test: Comparing Your Class to Others , 1999 .

[15]  Ralph L. Rosnow,et al.  Essentials of Behavioral Research: Methods and Data Analysis , 1984 .

[16]  Edward L. Deci,et al.  Intrinsic Motivation and Self-Determination , 2004 .

[17]  Ibrahim A. Halloun,et al.  Common sense concepts about motion , 1985 .

[18]  Erin M. Bardar,et al.  Development And Analysis Of Spectroscopic Learning Tools And The Light And Spectroscopy Concept Inventory For Introductory College Astronomy , 2006 .

[19]  Lei Baoa,et al.  Theoretical comparisons of average normalized gain calculations , 2006 .

[20]  D. Hestenes,et al.  A mechanics baseline test , 1992 .