Analysis of Individual Test Of Astronomy STandards (TOAST) Item Responses

The development of valid and reliable strategies to efficiently determine the knowledge landscape of introductory astronomy college students is an effort of great interest to the astronomy education community. This study examines individual item response rates from a widely used conceptual understanding survey, the Test Of Astronomy Standards (TOAST). The TOAST, a 27-item, multiple-choice format, criterion-referenced test, addresses both the full range of topics commonly taught in a one- or two-semester undergraduate introductory astronomy survey courses, and concepts described in various national science education standards, frameworks, and reform documents. The present study involves an examination of responses by 1104 participants, allowing for a rigorous item-by-item and distractor-by-distractor analysis of students’ responses. The results suggest that each individual TOAST item is functioning appropriately across a broad range of students, and has sufficient sensitivity to identify notable student misconceptions. These results also provide an opportunity to identify target areas of opportunity for astronomy education researchers that remain largely unstudied.

[1]  Philip M. Sadler,et al.  The Initial Knowledge State of High School Astronomy Students , 1992 .

[2]  Ngss Lead States.  Next generation science standards : for states, by states , 2013 .

[3]  Eve Kikas,et al.  Teachers' Conceptions and Misconceptions Concerning Three Natural Phenomena. , 2004 .

[4]  O Turkoglu,et al.  On pre-service science teachers preexisting knowledge levels about basic astronomy concepts , 2009 .

[5]  C. S. Wallace.  An investigation into introductory astronomy students' difficulties with cosmology, and the development, validation, and efficacy of a new suite of cosmology lecture-tutorials , 2011 .

[6]  Michael C. Rodriguez,et al.  A Review of Multiple-Choice Item-Writing Guidelines for Classroom Assessment , 2002 .

[7]  B. Partridge,et al.  Goals for "Astro 101": Report on Workshops for Department Leaders , 2003 .

[8]  A. Redfors,et al.  Swedish Upper Secondary Students’ Views of the Origin and Development of the Universe , 2006 .

[9]  Virginia A. Atwood,et al.  Preservice Elementary Teachers' Conceptions of the Causes of Seasons. , 1996 .

[10]  Tammy D. Lee,et al.  Redefining Curriculum Integration and Professional Development: In-service Teachers as Agents of Change , 2011 .

[11]  Colin S. Wallace,et al.  Do Concept Inventories Actually Measure Anything , 2010 .

[12]  Rebecca S. Lindell,et al.  Using the Lunar Phases Concept Inventory to Investigate College Students’ Pre‐instructional Mental Models of Lunar Phases , 2004 .

[13]  Douglas B. Clark,et al.  Exploring Sources of Variation in Studies of Knowledge Structure Coherence: Comparing Force Meanings and Force Meaning Consistency across Two Turkish Cities. , 2014 .

[14]  Janelle M. Bailey,et al.  Development Of A Concept Inventory To Assess Students’ Understanding And Reasoning Difficulties About The Properties And Formation Of Stars , 2006 .

[15]  William C. Philips.  Earth Science Misconceptions. , 1991 .

[16]  Frances P Lawrenz,et al.  Enhancing the Instructional Environment: Optimal Learning in Introductory Science Classes , 2005 .

[17]  Ann L. Brown,et al.  How people learn: Brain, mind, experience, and school. , 1999 .

[18]  Sharon Schleigh.  Impacting Society through Astronomy Undergraduate Courses , 2015 .

[19]  J. Baxter.  Children's understanding of familiar astronomical events , 1989 .

[20]  L. Agan.  Stellar Ideas: Exploring Students' Understanding of Stars. , 2004 .

[21]  Mark A. Francek,et al.  Assessing selected physical science and earth science misconceptions of middle school through university preservice teachers: Breaking the science 'misconception cycle' , 1994 .

[22]  Neil F. Comins.  Heavenly Errors: Misconceptions About the Real Nature of the Universe , 2001 .

[23]  D. Treagust,et al.  Secondary Students' Understanding of Gravity and the Motion of Planets. , 1989 .

[24]  Keith Skam,et al.  Determining Misconceptions about Astronomy. , 1994 .

[25]  M. Zeilik,et al.  Birth of the Astronomy Diagnostic Test: Prototest Evolution , 2001 .

[26]  R. Stiggins.  Student-centered classroom assessment , 1994 .

[27]  Timothy F. Slater,et al.  Development and Validation of the Light and Spectroscopy Concept Inventory , 2006 .

[28]  Matt Vassar,et al.  A Reliability Generalization Study of Coefficient Alpha for the UCLA Loneliness Scale , 2008, Journal of personality assessment.

[29]  J. Safko,et al.  Dynamics of a constructivist astronomy course for in-service teachers , 1996 .

[30]  Thomas A. Angelo,et al.  Classroom Assessment Techniques. A Handbook for Faculty. , 1988 .

[31]  S. Vosniadou.  Capturing and modeling the process of conceptual change. , 1994 .

[32]  W. Brewer,et al.  Mental models of the earth: A study of conceptual change in childhood , 1992, Cognitive Psychology.

[33]  J. K. Gilbert,et al.  A Method for Investigating Concept Understanding in Science , 1980 .

[34]  Douglas B. Clark,et al.  Constructed-Response as an Alternative to Interviews in Conceptual Change Studies: Students' Explanations of Force. , 2015 .

[35]  Julia D. Plummer,et al.  A Cross‐age Study of Children’s Knowledge of Apparent Celestial Motion , 2009 .

[36]  Timothy F. Slater,et al.  Impact of Backwards Faded Scaffolding in an Astronomy Course for Pre-service Elementary Teachers based on Inquiry , 2008 .

[37]  Timothy F. Slater,et al.  Hints of a Fundamental Misconception in Cosmology , 2002 .