A Primer for Developing Measures of Science Content Knowledge for Small-Scale Research and Instructional Use

This essay, intended for faculty involved in small-scale projects, courses, or educational research, provides a step-by-step guide to the process of developing, scoring, and validating content knowledge assessments. The authors illustrate their discussion with examples from their measures of high school students’ understanding of cell biology and epigenetics.

[1]  Kathrin F. Stanger-Hall,et al.  Multiple-Choice Exams: An Obstacle for Higher-Level Thinking in Introductory Science Classes , 2012, CBE life sciences education.

[2]  R. Glaser,et al.  Notebook Writing in Three Fifth-Grade Science Classrooms , 2001, The Elementary school journal.

[3]  Molly M. Stevens,et al.  Colloidal nanoparticles as advanced biological sensors , 2014, Science.

[4]  D. Cook,et al.  Current concepts in validity and reliability for psychometric instruments: theory and application. , 2006, The American journal of medicine.

[5]  Christopher J. Harris,et al.  Designing NGSS Assessments to Evaluate the Efficacy of Curriculum Interventions , 2013 .

[6]  Jerard Kehoe.  Basic Item Analysis for Multiple-Choice Tests. ERIC/AE Digest. , 1995 .

[7]  Melissa S. Yale,et al.  Differential Item Functioning , 2014 .

[8]  Joseph Krajcik,et al.  Supporting Grade 5-8 Students in Constructing Explanations in Science: The Claim, Evidence, and Reasoning Framework for Talk and Writing , 2011 .

[9]  Kevin A Hallgren,et al.  Computing Inter-Rater Reliability for Observational Data: An Overview and Tutorial. , 2012, Tutorials in quantitative methods for psychology.

[10]  M. Kane.  Validating the Interpretations and Uses of Test Scores , 2013 .

[11]  Mark Wilson,et al.  Constructing Measures: An Item Response Modeling Approach , 2004 .

[12]  A. Schuchat.  DEPARTMENT OF HEALTH & HUMAN SERVICES , 2015 .

[13]  Kathy Garvin-Doxas,et al.  Building, using, and maximizing the impact of concept inventories in the biological sciences: report on a National Science Foundation sponsored conference on the construction of concept inventories in the biological sciences. , 2007, CBE life sciences education.

[14]  E. Michael Nussbaum,et al.  Interview Procedures for Validating Science Assessments , 1997 .

[15]  Christine Y. O'Sullivan.  NAEP 1996 Science Report Card for the Nation and the States. Findings from the National Assessment of Educational Progress. , 1997 .

[16]  Gülnur Birol,et al.  Development of a Meiosis Concept Inventory , 2013, CBE life sciences education.

[17]  Harold P. Coyle,et al.  Assessing the Life Science Knowledge of Students and Teachers Represented by the K–8 National Science Standards , 2013, CBE life sciences education.

[18]  Ngss Lead States.  Next generation science standards : for states, by states , 2013 .

[19]  Derek C. Briggs,et al.  Diagnostic Assessment With Ordered Multiple-Choice Items , 2006 .

[20]  M. R. Espejo.  Applying the Rasch Model: Fundamental Measurement in the Human Sciences , 2004 .

[21]  Jonas Schmitt,et al.  Understanding By Design , 2016 .

[22]  Knut Neumann,et al.  Using Ordered Multiple-Choice Items To Assess Students’ Understanding of the Structure and Composition of Matter , 2013 .

[23]  Michael E. Martinez.  Cognition and the question of test item format , 1999 .

[24]  J. Leydens,et al.  Scoring Rubric Development: Validity and Reliability. , 2000 .

[25]  Robert Glaser,et al.  Investigating the Cognitive Complexity of Science Assessments , 1998 .

[26]  Todd D. Reeves,et al.  Contemporary Test Validity in Theory and Practice: A Primer for Discipline-Based Education Researchers , 2016, CBE life sciences education.

[27]  Rebecca M. Price,et al.  The EvoDevoCI: A Concept Inventory for Gauging Students’ Understanding of Evolutionary Developmental Biology , 2013, CBE life sciences education.

[28]  R. Glaser,et al.  Knowing What Students Know: The Science and Design of Educational Assessment , 2001 .

[29]  Judith A. Arter,et al.  Scoring Rubrics in the Classroom: Using Performance Criteria for Assessing and Improving Student Performance , 2000 .

[30]  James W. Pellegrino,et al.  Developing Assessments for the Next Generation Science Standards. , 2014 .

[31]  Maria Araceli Ruiz-Primo,et al.  Assessment and science education: Our essential new priority? , 2012 .

[32]  N. D. Dello Russo,et al.  Human subjects. , 2008, Journal of the American Dental Association.

[33]  Mark Nicolich,et al.  Developing a Measure of Scientific Literacy for Middle School Students. , 2014 .

[34]  Mark G. Simkin,et al.  Multiple-Choice Tests and Student Understanding: What Is the Connection? , 2005 .

[35]  Michael C. Rodriguez,et al.  A Review of Multiple-Choice Item-Writing Guidelines for Classroom Assessment , 2002 .

[36]  D. Hanauer,et al.  The Faculty Self-Reported Assessment Survey (FRAS): Differentiating Faculty Knowledge and Experience in Assessment , 2015, CBE life sciences education.

[37]  Steven E. Stemler,et al.  Best Practices in Interrater Reliability Three Common Approaches , 2008 .

[38]  M. Chi.  Quantifying Qualitative Analyses of Verbal Data: A Practical Guide , 1997 .

[39]  Daniel T. Hickey,et al.  Designing Assessments and Assessing Designs in Virtual Educational Environments , 2009 .

[40]  K. Bass,et al.  Using Small-Scale Randomized Controlled Trials to Evaluate the Efficacy of New Curricular Materials , 2014, CBE life sciences education.

[41]  Ross H. Nehm,et al.  A Critical Analysis of Assessment Quality in Genomics and Bioinformatics Education Research , 2013, CBE life sciences education.

[42]  Christine L. Moskalik,et al.  Development and Evaluation of a Genetics Literacy Assessment Instrument for Undergraduates , 2008, Genetics.

[43]  Jeffrey M. Perkel,et al.  LIFE SCIENCE TECHNOLOGIES: The Digital PCR Revolution , 2014 .

[44]  D. Eignor.  The standards for educational and psychological testing. , 2013 .

[45]  Deborah Allen,et al.  Rubrics: tools for making learning goals and evaluation criteria explicit for both teachers and learners. , 2006, CBE life sciences education.

[46]  David A. Gillam,et al.  A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas , 2012 .

[47]  Peggy Brickman,et al.  Best Practices for Measuring Students’ Attitudes toward Learning Science , 2013, CBE life sciences education.

[48]  M. Kline.  Teach , 2017 .

[49]  Richard J. Shavelson,et al.  Generalizability Theory: A Primer , 1991 .

[50]  Lawrence M. Rudner.  Questions To Ask When Evaluating Tests. , 1994 .

[51]  Gülnur Birol,et al.  Development of the Biological Experimental Design Concept Inventory (BEDCI) , 2014, CBE life sciences education.