Construct Validity of Self-Reported Metacognitive Learning Strategies

Despite their significant contributions to research on self-regulated learning, those favoring online and trace approaches have questioned the use of self-report to assess learners' use of learning strategies. An important rejoinder to such criticisms consists of examining the validity of self-report items. The present study was designed to assess the validity of items to assess 9th-grade students' use of planning, monitoring, and regulation when studying math. To establish response process evidence of construct validity, cognitive interviews were coded to determine whether students' interpretations of the items were consistent with their intended meaning and whether their response choices were congruent with those interpretations. Evidence supported the construct validity of monitoring and regulation items, but to a lesser degree those designed to assess planning. We discuss implications of the evidence for the self-report assessment of learners' use of metacognitive strategies.

[1]  P. Pintrich,et al.  Reliability and Predictive Validity of the Motivated Strategies for Learning Questionnaire (Mslq) , 1993 .

[2]  D. Eignor.  The standards for educational and psychological testing. , 2013 .

[3]  T. O. Nelson.  Metamemory: A Theoretical Framework and New Findings , 1990 .

[4]  J. Vermunt,et al.  Consistency and variability of learning strategies in different university courses , 1999 .

[5]  Philip H. Winne,et al.  Context moderates students' self-reports about how they study , 2001 .

[6]  M. Larsen,et al.  The Psychology of Survey Response , 2002 .

[7]  B. Zimmerman.  Attaining self-regulation: A social cognitive perspective. , 2000 .

[8]  G. Schellings,et al.  Applying learning strategy questionnaires: problems and possibilities , 2011 .

[9]  Philip H. Winne,et al.  Exploring students’ calibration of self reports about study tactics and achievement , 2002 .

[10]  M. Boekaerts,et al.  Self-Regulation in the Classroom: A Perspective on Assessment and Intervention , 2005 .

[11]  S. Messick.  Validity of Psychological Assessment: Validation of Inferences from Persons' Responses and Performances as Scientific Inquiry into Score Meaning. Research Report RR-94-45. , 1994 .

[12]  S. Karabenick,et al.  Motivation and students’ use of learning strategies: Evidence of unidirectional effects in mathematics classrooms , 2011 .

[13]  S. Tobias,et al.  Assessing Metacognitive Knowledge Monitoring , 1996 .

[14]  P. Pintrich.  A Manual for the Use of the Motivated Strategies for Learning Questionnaire (MSLQ). , 1991 .

[15]  W. McKeachie,et al.  The Making of the Motivated Strategies for Learning Questionnaire , 2005 .

[16]  Stuart A. Karabenick,et al.  Cognitive Processing of Self-Report Items in Educational Research: Do They Think What We Mean? , 2007 .

[17]  Evaluation of a Learning Strategies Scale for Middle School Students , 2009 .

[18]  Alija Kulenović,et al.  Standards for Educational and Psychological Testing , 1999 .

[19]  Michael E. Woolley,et al.  The Development and Evaluation of Procedures to Assess Child Self-Report Item Validity Educational and Psychological Measurement , 2006, Educational and psychological measurement.

[20]  S. Messick.  Validity of Psychological Assessment: Validation of Inferences from Persons' Responses and Performances as Scientific Inquiry into Score Meaning. Research Report RR-94-45. , 1994 .

[21]  Stuart A. Karabenick,et al.  Assessing Academic Self-Regulated Learning , 2005 .

[22]  Kristin L. K. Koskey,et al.  Cognitive validity of students’ self-reports of classroom mastery goal structure: What students are thinking and why it matters , 2010 .

[23]  Marcel V. J. Veenman,et al.  The assessment of metacognitive skills: what can be learned from multi-method designs? , 2005 .

[24]  P. Pintrich.  The role of goal orientation in self-regulated learning. , 2000 .

[25]  B. Hout-Wolters,et al.  Measuring strategy use with self-report instruments: theoretical and empirical considerations , 2011 .

[26]  C. Weinstein,et al.  Self-Regulation Interventions with a Focus on Learning Strategies , 2000 .

[27]  Philip H. Winne,et al.  Measuring Self-Regulated Learning , 2000 .

[28]  P. Pintrich,et al.  Motivational and self-regulated learning components of classroom academic performance. , 1990 .

[29]  B. Zimmerman.  Goal Setting: A Key Proactive Source of Academic Self-Regulation , 2012 .

[30]  Daniel L. Dinsmore,et al.  Focusing the Conceptual Lens on Metacognition, Self-regulation, and Self-regulated Learning , 2008 .

[31]  P. Pintrich.  Assessing Metacognition and Self-Regulated Learning , 2017 .

[32]  A. Brown.  Metacognition, executive control, self-regulation, and other more mysterious mechanisms , 1987 .

[33]  B. Zimmerman,et al.  Development of a Structured Interview for Assessing Student Use of Self-Regulated Learning Strategies , 1986 .