Performance Assessment of Critical Thinking: Conceptualization, Design, and Implementation

Enhancing students’ critical thinking (CT) skills is an essential goal of higher education. This article presents a systematic approach to conceptualizing and measuring CT. CT generally comprises the following mental processes: identifying, evaluating, and analyzing a problem; interpreting information; synthesizing evidence; and reporting a conclusion. We further posit that CT also involves dealing with dilemmas involving ambiguity or conflicts among principles and contradictory information. We argue that performance assessment provides the most realistic—and most credible—approach to measuring CT. From this conceptualization and construct definition, we describe one possible framework for building performance assessments of CT with attention to extended performance tasks within the assessment system. The framework is a product of an ongoing, collaborative effort, the International Performance Assessment of Learning (iPAL). The framework comprises four main aspects: (1) The storyline describes a carefully curated version of a complex, real-world situation. (2) The challenge frames the task to be accomplished (3). A portfolio of documents in a range of formats is drawn from multiple sources chosen to have specific characteristics. (4) The scoring rubric comprises a set of scales each linked to a facet of the construct. We discuss a number of use cases, as well as the challenges that arise with the use and valid interpretation of performance assessments. The final section presents elements of the iPAL research program that involve various refinements and extensions of the assessment framework, a number of empirical studies, along with linkages to current work in online reading and information processing.

[1]  Susanne Schmidt,et al.  Performance Assessment of Generic and Domain-Specific Skills in Higher Education Economics , 2020 .

[2]  J. Pellegrino,et al.  Education for Life and Work: Developing Transferable Knowledge and Skills in the 21st Century , 2013 .

[3]  Sam Wineburg,et al.  The Challenge That's Bigger than Fake News: Civic Reasoning in a Social Media Environment. , 2017 .

[4]  R. Shavelson,et al.  International Performance Assessment of Learning in Higher Education (iPAL): Research and Development , 2018 .

[5]  Richard J. Shavelson,et al.  Enhancing Scientific Thinking Through the Development of Critical Thinking in Higher Education , 2019, Redefining Scientific Thinking for Higher Education.

[6]  Julie Coiro,et al.  The New Literacies of Online Research and Comprehension: Assessing and Preparing Students for the 21st Century With Common Core State Standards , 2013 .

[7]  F. Oser,et al.  A Three-Level Model for Critical Thinking: Critical Alertness, Critical Reflection, and Critical Analysis , 2019, Frontiers and Advances in Positive Learning in the Age of InformaTiOn (PLATO).

[8]  Henry Braun,et al.  An Experimental Study of the Effects of Monetary Incentives on Performance on the 12th-Grade NAEP Reading Assessment , 2011, Teachers College Record: The Voice of Scholarship in Education.

[9]  Geneva D. Haertel,et al.  Evidence-Centered Design and Postsecondary Assessment , 2017 .

[10]  Ina V. S. Mullis,et al.  ePIRLS 2016: International Results in Online Informational Reading. , 2017 .

[11]  A. Mejía,et al.  Incorporating perspective analysis into critical thinking performance assessments. , 2019, The British journal of educational psychology.

[12]  Henry Braun,et al.  Performance assessment and standardization in higher education: A problematic conjunction? , 2019, The British journal of educational psychology.

[13]  T. Dary Erwin,et al.  Assessment of Critical Thinking: ETS's Tasks in Critical Thinking , 2003 .

[14]  D. Eignor The standards for educational and psychological testing. , 2013 .

[15]  Olga Zlatkin-Troitschanskaia,et al.  Advantages and challenges of performance assessment of student learning in higher education. , 2019, The British journal of educational psychology.

[16]  Roger Bolus,et al.  Evaluation Review the Collegiate Learning Assessment: Facts and Fantasies , 2007 .

[17]  Jacqueline P. Leighton,et al.  The risk-return trade-off: Performance assessments and cognitive validation of inferences. , 2019, The British journal of educational psychology.

[18]  Christiane Kuhn,et al.  Performance assessment to investigate the domain specificity of instructional skills among pre‐service and in‐service teachers of mathematics and economics , 2019, The British journal of educational psychology.

[19]  Robert J. Mislevy,et al.  Implications of Evidence‐Centered Design for Educational Testing , 2007 .

[20]  Richard J. Shavelson,et al.  Assessment of University Students’ Critical Thinking: Next Generation Performance Assessment , 2019, International Journal of Testing.

[21]  R. Shavelson,et al.  Assessment of Learning Outcomes in Higher Education , 2017 .

[22]  Ou Lydia Liu,et al.  Assessing Critical Thinking in Higher Education: Current State and Directions for Next-Generation Assessment. Research Report. ETS RR-14-10. , 2014 .

[23]  Stephen M. Kosslyn,et al.  Building the Intentional University: Minerva and the Future of Higher Education , 2017 .

[24]  R. Almond,et al.  A BRIEF INTRODUCTION TO EVIDENCE-CENTERED DESIGN , 2003 .

[25]  J. Paris Performance assessment. , 1998, Journal of public health medicine.

[26]  Doris Zahner,et al.  Reliability and Validity of CLA + , 2016 .

[27]  Patricia H. Wheeler,et al.  Resource Handbook on Performance Assessment and Measurement: A Tool for Students, Practitioners, and Policymakers. , 1993 .

[28]  Richard J. Shavelson,et al.  On an Approach to Testing and Modeling Competence , 2013 .

[29]  S. Messick The Interplay of Evidence and Consequences in the Validation of Performance Assessments , 1994 .

[30]  Kevin Eagan,et al.  Undergraduate Teaching Faculty: The HERI Faculty Survey 2016–2017 , 2019 .

[31]  Auli Toom,et al.  Developing a performance assessment task in the Finnish higher education context: Conceptual and empirical insights. , 2019, The British journal of educational psychology.

[32]  H. Laborit,et al.  [Experimental study]. , 1958, Bulletin mensuel - Societe de medecine militaire francaise.

[33]  S. Freud In Over Our Heads. The Mental Demands of Modern Life , 1997 .

[34]  D. Mcclelland Testing for competence rather than for "intelligence". , 1973, The American psychologist.

[35]  Sam Wineburg,et al.  Evaluating information: The cornerstone of civic online reasoning , 2016 .

[36]  Susanne Schmidt,et al.  On the complementarity of holistic and analytic approaches to performance assessment scoring. , 2019, The British journal of educational psychology.