Quantifying critical thinking: Development and validation of the physics lab inventory of critical thinking

Introductory physics lab instruction is undergoing a transformation, with increasing emphasis on developing experimentation and critical thinking skills. These changes present a need for standardized assessment instruments to determine the degree to which students develop these skills through instructional labs. In this article, we present the development and validation of the Physics Lab Inventory of Critical thinking (PLIC). We define critical thinking as the ability to use data and evidence to decide what to trust and what to do. The PLIC is a 10-question, closed-response assessment that probes student critical thinking skills in the context of physics experimentation. Using interviews and data from 5584 students at 29 institutions, we demonstrate, through qualitative and quantitative means, the validity and reliability of the instrument at measuring student critical thinking skills. This establishes a valuable new assessment instrument for instructional labs.

[1]  K. Slaughter.  Mapping the transition : content and pedagogy from school through university , 2012 .

[2]  Gülnur Birol,et al.  Development of the Biological Experimental Design Concept Inventory (BEDCI) , 2014, CBE life sciences education.

[3]  Rodney L. Doran,et al.  Basic measurement and evaluation of science instruction , 1980 .

[4]  N. Holmes,et al.  Teaching critical thinking , 2015, Proceedings of the National Academy of Sciences.

[5]  Richard T. White,et al.  The link between the laboratory and learning , 1996 .

[6]  Sarah B. McKagan,et al.  Gender gap on concept inventories in physics: what is consistent, what is inconsistent,and what factors influence the gap? , 2013, 1307.0912.

[7]  Rebecca Lippmann.  Students' understanding of measurement and uncertainty in the physics laboratory: Social construction, underlying concepts, and quantitative analysis , 2003 .

[8]  Stef van Buuren,et al.  MICE: Multivariate Imputation by Chained Equations in R , 2011 .

[9]  H. J. Lewandowski,et al.  Students' epistemologies about experimental physics: Validating the Colorado Learning Attitudes about Science Survey for Experimental Physics , 2015, 1512.06869.

[10]  Vincent N. Lunetta,et al.  The Laboratory in Science Education: Foundations for the Twenty-First Century , 2004 .

[11]  Wendy Kristine Adams.  Development of a problem solving evaluation instrument; untangling of specific problem solving assets , 2007 .

[12]  Bob Campbell,et al.  First‐year physics students’ perceptions of the quality of experimental measurements , 1998 .

[13]  Carol S. Dweck,et al.  Motivational processes affecting learning. , 1986 .

[14]  Saalih Allie,et al.  The development of first year physics students' ideas about measurement in terms of point and set paradigms , 2001 .

[15]  Michelle K. Smith,et al.  Active learning increases student performance in science, engineering, and mathematics , 2014, Proceedings of the National Academy of Sciences.

[16]  P. Brickman,et al.  Developing a Test of Scientific Literacy Skills (TOSLS): Measuring Undergraduates’ Evaluation of Scientific Information and Arguments , 2012, CBE life sciences education.

[17]  Ada Haynes,et al.  Assessing Critical Thinking in STEM and Beyond , 2007 .

[18]  Carl E. Wieman,et al.  Assessing modeling in the lab: Uncertainty and measurement , 2015 .

[19]  Ben Van Dusen,et al.  Missing data and bias in physics education research: A case for using multiple imputation , 2018, Physical Review Physics Education Research.

[20]  N. Holmes,et al.  Value added or misattributed? A multi-institution study on the educational benefit of labs for reinforcing physics content , 2017, 1705.03580.

[21]  James Day,et al.  Development of the Concise Data Processing Assessment , 2011 .

[22]  Jennifer Docktor,et al.  Synthesis of Discipline-Based Education Research in Physics. , 2014 .

[23]  Bethany R. Wilcox,et al.  Improvement or selection? A longitudinal analysis of students' views about experimental physics in their lab courses , 2017, 1706.03681.

[24]  D. Rubin.  INFERENCE AND MISSING DATA , 1975 .

[25]  Andy Buffler,et al.  The Relationship between Students’ Views of the Nature of Science and their Views of the Nature of Scientific Measurement , 2009 .

[26]  Karen Sirum,et al.  The Experimental Design Ability Test (EDAT) , 2011 .

[27]  Bethany R. Wilcox,et al.  Developing skills versus reinforcing concepts in physics labs: Insight from a survey of students' beliefs about experimental physics , 2017 .

[28]  D. Willingham.  Critical Thinking: Why Is It So Hard to Teach? , 2008 .

[29]  S. Nicholson,et al.  Roster of Physics Departments with Enrollment and Degree Data, 2014: Results from the 2014 Survey of Enrollments and Degrees. Focus On. , 2012 .

[30]  N. Holmes,et al.  Measuring the Impact of Introductory Physics Labs on Learning , 2015, 1507.00264.

[31]  R. Hake.  Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses , 1998 .

[32]  Avi Hofstein,et al.  The Role of the Laboratory in Science Teaching: Neglected Aspects of Research , 1982 .

[33]  Quantitative Comparisons to Promote Inquiry in the Introductory Physics Lab , 2015, 1506.02967.

[34]  P. Heron,et al.  Preparing physics students for 21st-century careers , 2017 .

[35]  Anton E. Lawson,et al.  The development and validation of a classroom test of formal reasoning , 1978 .

[36]  Benjamin M. Zwickl,et al.  Incorporating learning goals about modeling into an upper-division physics laboratory experiment , 2013, 1301.4414.

[37]  Robert J. Beichner,et al.  Approaches to Data Analysis of Multiple-Choice Questions. , 2009 .

[38]  N. Holmes,et al.  Interview validation of the Physics Lab Inventory of Critical thinking (PLIC) , 2018, 1802.02424.

[39]  R. Millar.  The role of practical work in the teaching and learning of science , 2004 .

[40]  S. Nicholson,et al.  Physics Enrollments: Results from the 2008 Survey of Enrollments and Degrees. Focus On. , 2011 .

[41]  Bethany R. Wilcox,et al.  Alternative model for the administration and analysis of research-based assessments , 2016, 1601.07896.

[42]  Bethany R. Wilcox,et al.  Developing skills versus reinforcing concepts in physics labs: Insight from a survey of students' beliefs about experimental physics , 2016, 1611.02322.

[43]  C. Dweck.  Self-Theories: Their Role in Motivation, Personality, and Development. Essays in Social Psychology. , 1999 .

[44]  Eleanor W. Close,et al.  Participation and performance on paper- and computer-based low-stakes assessments , 2017, International Journal of STEM Education.

[45]  Benjamin M. Zwickl,et al.  Model-Based Reasoning in the Physics Laboratory: Framework and Initial Results. , 2015 .

[46]  Marie-Geneviève Séré,et al.  Towards renewed research questions from the outcomes of the European project Labwork in Science Education , 2002 .

[47]  Saalih Allie,et al.  Impact of a conventional introductory laboratory course on the understanding of measurement , 2008 .

[48]  Carl E. Wieman,et al.  Comparative Cognitive Task Analyses of Experimental Science and Instructional Laboratory Courses. , 2015 .

[49]  Steve Olson,et al.  Engage to Excel: Producing One Million Additional College Graduates with Degrees in Science, Technology, Engineering, and Mathematics. Report to the President. , 2012 .

[50]  Abigail M. Bogdan,et al.  Reasoning with alternative explanations in physics: The cognitive accessibility rule , 2018 .

[51]  Sarah B. McKagan,et al.  How physics instruction impacts students' beliefs about learning physics , 2014 .

[52]  Starr Nicholson,et al.  Roster of Physics Departments with Enrollment and Degree Data, 2002. AIP Report. , 2001 .

[53]  Natasha G. Holmes,et al.  Structured quantitative inquiry labs : developing critical thinking in the introductory physics laboratory , 2014 .

[54]  A. C. Butler.  Multiple-Choice Testing in Education: Are the Best Practices for Assessment Also Good for Learning? , 2018, Journal of Applied Research in Memory and Cognition.

[55]  Peter A. Facione,et al.  Using the California Critical Thinking Skills Test in Research, Evaluation, and Assessment. , 1991 .

[56]  Susan R. Singer,et al.  Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering. , 2012 .

[57]  S. van Buuren.  Flexible Imputation of Missing Data, Second Edition , 2018 .

[58]  Duane L. Deardorff,et al.  Introductory physics students' treatment of measurement uncertainty , 2001 .