The development of a questionnaire on metacognition for students in higher education

Background Interest in the role of metacognition has been steadily rising in most forms of education. This study focuses on the construction of a questionnaire for measuring metacognitive knowledge, metacognitive regulation and metacognitive responsiveness among students in higher education and the subsequent process of testing to determine its validity. Purpose The aim of the study was to construct an original instrument for measuring features of metacognition, henceforth referred to as the Awareness of Independent Learning Inventory (AILI), and further to establish the similarities and differences between this model and existing instruments for measuring metacognition. Sample The AILI questionnaire was distributed to 1058 students in various types of Teacher Training Institutes in the Netherlands and Belgium. The abridged English version of the questionnaire was administered to another sample of 729 students reading Economics and Business Administration at the University of Maastricht in the south of the Netherlands. Design and methods The AILI instrument was constructed on the basis of a facet design along two dimensions: components of metacognition and topics of concern to students in higher education. The data gathered with the instrument was analyzed by means of a generalisability study and a decision study, respectively. The validity of the instrument was investigated by using confirmatory factor analysis. Results The generalisability study showed that the reliability of the instrument was satisfactory. The decision study revealed that the number of items included in the questionnaire could be reduced substantially by leaving out two components of one of the dimensions in the facet design, without losing too much generalisability. The validity study showed that there was a considerable level of congruity between parts of the AILI questionnaire and the relevant parts of the Motivated Strategies for Learning Questionnaire (MSLQ). Conclusions The AILI questionnaire is a reliable and valid instrument for measuring metacognitive knowledge, regulation and responsiveness. It is suitable for use in the evaluation of the effects of interventions that purport to increase metacognitive knowledge, regulation and responsiveness of students in higher education.

[1]  Nigel O'Brian,et al.  Generalizability Theory I , 2003 .

[2]  W. Hofstee.  Will the True Trait Theorist Please Stand Up , 1994 .

[3]  W. McKeachie,et al.  The Making of the Motivated Strategies for Learning Questionnaire , 2005 .

[4]  Paul C. Burnett,et al.  A Comparison of Conventional Versus Confirmatory Factor Analytic Techniques for Validating Existing Scales , 1997 .

[5]  M. Elshout-Mohr.  Metacognitie van lerenden in onderwijsleerprocessen. , 2000 .

[6]  B. Hout-Wolters.  Assessing Active Self-directed Learning , 2000 .

[7]  Gregory Schraw,et al.  Assessing metacognitive awareness , 1994 .

[8]  Franciscus Petrus Christoffel Maria de Jong.  Zelfstandig leren : regulatie van het leerproces en leren reguleren: een procesbenadering , 1992 .

[9]  B. V. van Hout-Wolters,et al.  Multi-domain, multi-method measures of metacognitive activity: what is all the fuss about metacognition … indeed? , 2012 .

[10]  P. Winne,et al.  Feedback and Self-Regulated Learning: A Theoretical Synthesis , 1995 .

[11]  Paul C. Burnett,et al.  Conventional versus confirmatory factor analysis : methods for validating the structure of existing scales , 1997 .

[12]  B. Zimmerman.  Self-Regulated Learning and Academic Achievement: An Overview , 1990 .

[13]  M. Veenman,et al.  Intelligence and metacognitive skillfulness in secondary education. , 2001 .

[14]  Donald B. Rubin,et al.  The Dependability of Behavioral Measurements: Theory of Generalizability for Scores and Profiles. , 1974 .

[15]  Marcel V. J. Veenman,et al.  The assessment of metacognitive skills: what can be learned from multi-method designs? , 2005 .

[16]  J. Elshout,et al.  Changes in the relation between cognitive and metacognitive skills during the acquisition of expertise , 1999 .

[17]  M. van Daalen-Kapteijns,et al.  Functions of an electronic portfolio in a constructivistic learning environment , 2000 .

[18]  D. Cross,et al.  Informed strategies for learning: a program to improve children's reading awareness and comprehension , 1984 .

[19]  Bruce C. Howard,et al.  Metacognition and Self-Regulated Learning Constructs , 2004 .

[20]  Ann L. Brown,et al.  Guided, Cooperative Learning and Individual Knowledge Acquisition , 2018, Knowing, Learning, and Instruction.

[21]  J. Flavell.  Speculations about the nature and development of metacognition , 1987 .

[22]  B. V. van Hout-Wolters,et al.  Metacognitive activities in text-studying and problem-solving: Development of a taxonomy , 2006 .

[23]  Stimulating independent learning: a quasi‐experimental study on portfolio , 2008 .

[24]  Arno Libotton,et al.  Portfolio Assignments in Teacher Education: A Tool For Self-regulating the Learning Process? , 2007 .

[25]  M. Elshout-Mohr,et al.  Functies van een elektronisch portfolio in een constructivistische leeromgeving , 2000 .

[26]  Gregory Schraw.  Promoting general metacognitive awareness , 1998 .

[27]  D. Schoen.  The Reflective Practitioner , 1983 .

[28]  Jay McTighe,et al.  Assessing Student Outcomes: Performance Assessment Using the Dimensions of Learning Model , 1993 .

[29]  Michael J. Middleton,et al.  Turning the Kaleidoscope: What We See When Self-Regulated Learning is Viewed With a Qualitative Lens , 2002 .

[30]  L. Resnick,et al.  Knowing, Learning, and Instruction , 2018 .

[31]  P. Nardi.  Doing Survey Research: A Guide to Quantitative Methods , 2005 .

[32]  P. van Petegem,et al.  Portfolio as a means of promoting autonomous learning in teacher education: a quasi-experimental study , 2008 .

[33]  P. Pintrich,et al.  Reliability and Predictive Validity of the Motivated Strategies for Learning Questionnaire (Mslq) , 1993 .

[34]  J. Elshout,et al.  The generality vs domain-specificity of metacognitive skills in novice learning across domains. , 1997 .

[35]  Fred Paas,et al.  Reflection prompts and tutor feedback in a web-based learning environment: effects on students' self-regulated learning competence , 2004, Comput. Hum. Behav..

[36]  A. Brown.  Metacognition, executive control, self-regulation, and other more mysterious mechanisms , 1987 .

[37]  D. Schoen,et al.  The Reflective Practitioner: How Professionals Think in Action , 1985 .

[38]  John T. E. Richardson,et al.  Student Learning: Research in Education and Cognitive Psychology , 1987 .

[39]  Jamal Abedi,et al.  Reliability and Validity of a State Metacognitive Inventory: Potential for Alternative Assessment , 1996 .

[40]  Eva Nick,et al.  The dependability of behavioral measurements: theory of generalizability for scores and profiles , 1973 .

[41]  H. Walberg,et al.  Toward a Knowledge Base for School Learning , 1993 .

[42]  Anthony J. Onwuegbuzie,et al.  Characteristics of respondents who respond differently to positively and negatively worded items on rating scales , 2003 .

[43]  Donald A. Schön.  The reflective practitioner : how professionals think in action , 1986 .

[44]  Charles D. Spielberger,et al.  Anxiety: Current Trends in Theory and Research , 1972 .

[45]  Allison Littlejohn,et al.  Measuring Self‐Regulated Learning in the Workplace , 2015, SSRN Electronic Journal.

[46]  Wynetta Y. Lee.  Assessment and evaluation in higher education , 2010 .

[47]  J. Flavell.  Metacognition and Cognitive Monitoring: A New Area of Cognitive-Developmental Inquiry. , 1979 .