Development of the Biological Variation In Experimental Design And Analysis (BioVEDA) assessment

Variation is an important concept that underlies experimental design and data analysis. Incomplete understanding of variation can preclude students from designing experiments that adequately manage organismal and experimental variation, and from accurately conducting and interpreting statistical analyses of data. Because of the lack of assessment instruments that measure students’ ideas about variation in the context of biological investigations, we developed the Biological Variation in Experimental Design and Analysis (BioVEDA) assessment. Psychometric analyses indicate that BioVEDA assessment scores are reliable/precise. We provide evidence that the BioVEDA instrument can be used to evaluate students’ understanding of biological variation in the context of experimental design and analysis relative to other students and to their prior scores.

[1]  J. Batzli,et al.  Crossing the Threshold: Bringing Biological Variation to the Foreground , 2016, CBE life sciences education.

[2]  Wim Van Den Noortgate,et al.  Students’ misconceptions of statistical inference: A review of the empirical evidence from research on statistics education , 2007 .

[3]  Alexander Pollatsek,et al.  Understanding the effects of sample size on the variability of the mean , 1990 .

[4]  Theresa J. B. Kline Modern Test Theory: Assumptions, Equations, Limitations, and Item Analyses , 2005 .

[5]  Naomi S. Altman,et al.  Points of significance: Sources of variation , 2014, Nature Methods.

[6]  Robert J. Beichner,et al.  Approaches to Data Analysis of Multiple-Choice Questions. , 2009 .

[7]  S. Glantz Biostatistics: how to detect, correct and prevent errors in the medical literature. , 1980, Circulation.

[8]  Louisa A. Stark,et al.  A Primer for Developing Measures of Science Content Knowledge for Small-Scale Research and Instructional Use , 2016, CBE life sciences education.

[9]  Philip Osteen An Introduction to Using Multidimensional Item Response Theory to Assess Latent Factor Structures , 2010, Journal of the Society for Social Work and Research.

[10]  S. Goodman A dirty dozen: twelve p-value misconceptions. , 2008, Seminars in hematology.

[11]  Susan Jo Russell,et al.  Children's Concepts of Average and Representativeness. , 1995 .

[12]  Yves Rosseel,et al.  lavaan: An R Package for Structural Equation Modeling , 2012 .

[13]  J. Batzli,et al.  Beyond Punnett Squares: Student Word Association and Explanations of Phenotypic Variation through an Integrative Quantitative Genetics Unit Investigating Anthocyanin Inheritance and Expression in Brassica rapa Fast Plants , 2014, CBE life sciences education.

[14]  Gülnur Birol,et al.  Development of the Biological Experimental Design Concept Inventory (BEDCI) , 2014, CBE life sciences education.

[15]  Sharon J. Lane-Getaz Development of a Reliable Measure of Students' Inferential Reasoning Ability. , 2013 .

[16]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[17]  Lei Bao,et al.  Analyzing force concept inventory with item response theory , 2010 .

[18]  John Hattie,et al.  Procedures for Assessing the Validities of Tests Using the "Known-Groups" Method , 1984 .

[19]  Zemira R. Mevarech A deep structure model of students' statistical misconceptions , 1983 .

[20]  Richard Lehrer,et al.  Supporting the Development of Conceptions of Statistics by Engaging Students in Measuring and Modeling Variability , 2007, Int. J. Comput. Math. Learn..

[21]  Jennifer Knight,et al.  Harnessing Technology to Improve Formative Assessment of Student Conceptions in STEM: Forging a National Network , 2011, CBE life sciences education.

[22]  Karen Sirum,et al.  The Experimental Design Ability Test (EDAT) , 2011 .

[23]  T. Kline Classical Test Theory: Assumptions, Equations, Limitations, and Item Analyses , 2005 .

[24]  P. Gagné,et al.  Measurement Model Quality, Sample Size, and Solution Propriety in Confirmatory Factor Models , 2006, Multivariate behavioral research.

[25]  William J. Boone,et al.  Rasch Analysis for Instrument Development: Why, When, and How? , 2016, CBE life sciences education.

[26]  Edward F. Redish,et al.  Language of Physics, Language of Math: Disciplinary Culture and Dynamic Epistemology , 2014 .

[27]  N. Schmitt Uses and abuses of coefficient alpha. , 1996 .

[28]  K. Taber The Use of Cronbach’s Alpha When Developing and Reporting Research Instruments in Science Education , 2017, Research in Science Education.

[29]  C. Wild,et al.  Statistical Thinking in Empirical Enquiry , 1999 .

[30]  R. Philip Chalmers,et al.  mirt: A Multidimensional Item Response Theory Package for the R Environment , 2012 .

[31]  Andrew Shtulman,et al.  The Relation Between Essentialist Beliefs and Evolutionary Reasoning , 2008, Cogn. Sci..

[32]  Richard Lehrer,et al.  Modeling Natural Variation Through Distribution , 2004 .

[33]  G. Birol,et al.  Development of the Statistical Reasoning in Biology Concept Inventory (SRBCI) , 2016, CBE life sciences education.

[34]  Richard Lehrer,et al.  Seeding Evolutionary Thinking by Engaging Children in Modeling Its Foundations. , 2012 .

[35]  Patrick Onghena,et al.  How Confident are Students in their Misconceptions about Hypothesis Tests? , 2009 .

[36]  Trevor R Anderson,et al.  Development and Validation of a Rubric for Diagnosing Students’ Experimental Design Knowledge and Difficulties , 2014, CBE life sciences education.

[37]  Jose P. Mestre,et al.  Toward a Model of Transfer as Sense-Making , 2013 .

[38]  K. Pesudovs,et al.  Rasch analysis reveals problems with multiplicative scoring in the macular disease quality of life questionnaire. , 2012, Ophthalmology.

[39]  Ross H. Nehm,et al.  A Critical Analysis of Assessment Quality in Genomics and Bioinformatics Education Research , 2013, CBE life sciences education.

[40]  Jennifer L. Momsen,et al.  Introductory Biology Students’ Conceptual Models and Explanations of the Origin of Variation , 2014, CBE life sciences education.

[41]  Tammy M. Long,et al.  Analyzing change in students' gene-to-evolution models in college-level introductory biology , 2013 .

[42]  Yan Liu,et al.  EXPLORING STUDENTS’ CONCEPTIONS OF THE STANDARD DEVIATION , 2005 .

[43]  Beth Chance,et al.  Reasoning about Sampling Distribitions , 2004 .

[44]  Beth Chance,et al.  ASSESSING STUDENTS’ CONCEPTUAL UNDERSTANDING AFTER A FIRST COURSE IN STATISTICS , 2007 .

[45]  Ivo D. Dinov,et al.  Law of Large Numbers: The Theory, Applications and Technology-Based Education , 2009, Journal of statistics education : an international journal on the teaching and learning of statistics.

[46]  Anneke M Metz,et al.  Teaching statistics in biology: using inquiry-based learning to strengthen understanding of statistical analysis in biology laboratory courses. , 2008, CBE life sciences education.

[47]  Robyn Reaburn,et al.  INTRODUCTORY STATISTICS COURSE TERTIARY STUDENTS’ UNDERSTANDING OF P-VALUES , 2014, STATISTICS EDUCATION RESEARCH JOURNAL.

[48]  D. Kuhn,et al.  Is Developing Scientific Thinking All About Learning to Control Variables? , 2005, Psychological science.

[49]  Development of a Biological Science Quantitative Reasoning Exam (BioSQuaRE) , 2017, CBE life sciences education.

[50]  Michael C. Rodriguez,et al.  A Review of Multiple-Choice Item-Writing Guidelines for Classroom Assessment , 2002 .

[51]  Naomi S. Altman,et al.  Points of Significance: Visualizing samples with box plots , 2014, Nature Methods.

[52]  Erin L. Dolan,et al.  Recognizing Students’ Scientific Reasoning: A Tool for Categorizing Complexity of Reasoning During Teaching by Inquiry , 2010, Journal of science teacher education.

[53]  C. Velozo,et al.  Measurement Qualities of a Self-Report and Therapist-Scored Functional Capacity Instrument Based on the Dictionary of Occupational Titles , 2006, Journal of Occupational Rehabilitation.

[54]  A. Collins,et al.  Situated Cognition and the Culture of Learning , 1989 .

[55]  G. Masters,et al.  Rating scale analysis , 1982 .

[56]  Martin Krzywinski,et al.  Points of Significance: Error bars , 2013, Nature Methods.

[57]  R. Millar,et al.  Reasoning from data: How students collect and interpret data in science investigations , 2004 .

[58]  Alija Kulenović,et al.  Standards for Educational and Psychological Testing , 1999 .

[59]  Gregory A. Moyerbrailean,et al.  1, 2, 3, 4: Infusing Quantitative Literacy into Introductory Biology , 2010, CBE life sciences education.