Development and Validation of the Star Properties Concept Inventory

Concept inventories (CIs)—typically multiple-choice instruments that focus on a single or small subset of closely related topics—have been used in science education for more than a decade. This paper describes the development and validation of a new CI for astronomy, the Star Properties Concept Inventory (SPCI). Questions cover the areas of stellar properties (focussing primarily on mass, temperature, luminosity, and lifetime), nuclear fusion, and star formation. Distracters were developed from known alternative conceptions and reasoning difficulties commonly held by students. The SPCI was tested through an iterative process where different testing formats (open-ended, multiple-choice + explain, and multiple-choice) were compared to ensure that the distracters were in fact the most common among the testing population. Content validity was established through expert reviews by 26 astronomy instructors. The SPCI Version 3 was then tested in multiple introductory undergraduate astronomy courses for non-science majors. Post-test scores (out of 23 possible) were significantly greater (M = 11.8, SD = 3.87) than the pre-test scores (M = 7.09, SD = 2.73). The low post-test score—only 51.3%—could indicate a need for changing instructional strategies on the topics of stars and star formation.

[1]  E. Prather,et al.  College Students' Preinstructional Ideas About Stars and Star Formation , 2009 .

[2]  Imelda S. Caleon,et al.  Development and Application of a Three‐Tier Diagnostic Test to Assess Secondary Students’ Understanding of Waves , 2010 .

[3]  Edward E. Prather,et al.  A national study assessing the teaching and learning of introductory astronomy. Part I. The effect of interactive instruction , 2009 .

[4]  D. Hestenes,et al.  A mechanics baseline test , 1992 .

[5]  Rebecca S. Lindell,et al.  Developing the Lunar Phases Concept Inventory , 2002 .

[6]  Beth Hufnagel,et al.  Pre-course Results from the Astronomy Diagnostic Test , 2000, Publications of the Astronomical Society of Australia.

[7]  Concept Inventories for ASTRO 101 , 2009 .

[8]  B. Partridge,et al.  Goals for "Astro 101": Report on Workshops for Department Leaders , 2003 .

[9]  Andrew Fraknoi.  Enrollments in Astronomy 101 Courses , 2001 .

[10]  M. Zeilik,et al.  Birth of the Astronomy Diagnostic Test: Prototest Evolution , 2001 .

[11]  R. Hake.  Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses , 1998 .

[12]  Edward E. Prather,et al.  Teaching and learning astronomy in the 21st century , 2009 .

[13]  S. Embretson,et al.  Item response theory for psychologists , 2000 .

[14]  Timothy F. Slater,et al.  Conducting astronomy education research : a primer , 2011 .

[15]  J. Libarkin,et al.  Assessment of Learning in Entry-Level Geoscience Courses: Results from the Geoscience Concept Inventory , 2005 .

[16]  Timothy F. Slater,et al.  Analysis of the Astronomy Diagnostic Test , 2007 .

[17]  Edward E. Prather,et al.  Development of the Solar System Concept Inventory , 2009 .

[18]  A. Strauss,et al.  The discovery of grounded theory: strategies for qualitative research aldine de gruyter , 1968 .

[19]  J. Keller.  Part I. Development of a concept inventory addressing students' beliefs and reasoning difficulties regarding the greenhouse effect, Part II. Distribution of chlorine measured by the Mars Odyssey Gamma Ray Spectrometer , 2006 .

[20]  Michael C. LoPresto,et al.  Using the Star Properties Concept Inventory to compare instruction with lecture tutorials to traditional lectures , 2009 .

[21]  Nicholas M. Schneider,et al.  The Cosmic Perspective , 1998 .

[22]  Darren George,et al.  SPSS for Windows Step by Step: A Simple Guide and Reference , 1998 .

[23]  Philip M. Sadler,et al.  The Astronomy and Space Science Concept Inventory: Development and Validation of Assessment Instruments Aligned with the K-12 National Science Standards , 2009 .

[24]  Timothy F. Slater,et al.  The Need for a Light and Spectroscopy Concept Inventory for Assessing Innovations in Introductory Astronomy Survey Courses , 2005 .

[25]  Robert J. Beichner,et al.  Approaches to Data Analysis of Multiple-Choice Questions. , 2009 .

[26]  Ibrahim A. Halloun,et al.  The initial knowledge state of college physics students , 1985 .

[27]  Timothy F. Slater,et al.  Development and Validation of the Light and Spectroscopy Concept Inventory , 2006 .

[28]  Janelle M. Bailey,et al.  Development Of A Concept Inventory To Assess Students’ Understanding And Reasoning Difficulties About The Properties And Formation Of Stars , 2006 .

[29]  Grace L. Deming.  Results from the Astronomy Diagnostic Test National Project , 2001 .

[30]  T. Bruce,et al.  Understanding Reliability and Coefficient alpha, Really , 2003 .

[31]  Timothy F. Slater,et al.  Research on a Lecture-Tutorial Approach to Teaching Introductory Astronomy for Non-Science Majors , 2004 .

[32]  D. Bruning.  2006 Survey of Introductory Astronomy Textbooks , 2005 .

[33]  R. H. Garstang,et al.  Astronomy, the evolving universe , 1976 .

[34]  D. Treagust.  Development and use of diagnostic tests to evaluate students’ misconceptions in science , 1988 .

[35]  B. Hufnagel,et al.  Development of the Astronomy Diagnostic Test , 2001 .

[36]  Grace L. Deming,et al.  Who's taking ASTRO 101? , 2001 .

[37]  M. Rollnick,et al.  Big Ideas: A review of astronomy education research 1974–2008 , 2010 .

[38]  Timothy F. Slater,et al.  A Review of Astronomy Education Research , 2003 .

[39]  Rebecca S. Lindell,et al.  Are They All Created Equal? A Comparison of Different Concept Inventory Development Methodologies , 2007 .

[40]  P. Fayers.  Item Response Theory for Psychologists , 2004, Quality of Life Research.

[41]  J. Nunnally,et al.  Psychometric Theory: NY. , 1978 .

[43]  I. Seidman.  Interviewing as qualitative research : a guide for researchersin education and the social sciences , 1991 .

[44]  M. J. Allen.  Introduction to Measurement Theory , 1979 .

[45]  David F. Treagust,et al.  Evaluating students' misconceptions by means of diagnostic multiple choice items , 1986 .

[46]  Erin M. Bardar,et al.  Development And Analysis Of Spectroscopic Learning Tools And The Light And Spectroscopy Concept Inventory For Introductory College Astronomy , 2006 .

[47]  Steven J. Osterlind,et al.  Modern Measurement: Theory, Principles, and Applications of Mental Appraisal , 2005 .

[48]  Edward E. Prather,et al.  A National Study Assessing the Teaching and Learning of Introductory Astronomy Part II: The Connection between Student Demographics and Learning , 2010 .

[49]  G. Norman,et al.  Development and Evaluation of the Conceptual Inventory of Natural Selection , 2002 .

[50]  E. Prather,et al.  Student Understanding of Ionizing Radiation and Radioactivity. , 2001 .

[51]  Timothy F. Slater,et al.  What topics are taught in introductory astronomy courses , 2001 .

[52]  Frances P Lawrenz,et al.  Enhancing the Instructional Environment: Optimal Learning in Introductory Science Classes , 2005 .

[53]  Colin S. Wallace,et al.  Do Concept Inventories Actually Measure Anything , 2010 .

[54]  Rebecca Susan Lindell.  Enhancing college students' understanding of lunar phases , 2001 .

[55]  D. Bruning.  Survey of Introductory Astronomy Textbooks: An Update , 2006 .

[56]  D. George,et al.  SPSS for Windows step by step , 1998 .