The effect of feedback on metacognition - A randomized experiment using polling technology

Abstract This study explores the effects of formative feedback on students' metacognitive skills when using feedback strategies with polling technology. Using a randomized field experiment among 633 physics students in six schools in Dutch secondary education, we study assessments with the polling technology Socrative, by dividing students into three groups. Students in the cooperative group use a combination of peer discussions and teacher feedback, while students in the individual group use teacher feedback. To compare differences in metacognitive skills, students in the control group only use Socrative, but do not receive formative feedback from either teacher or peers. The results show that there is a significant positive effect of the cooperative treatment on both metacognitive skills and motivation in comparison with the control group. We find that students with low metacognitive skills benefit significantly more from the cooperative treatment than students with high metacognitive skills. No effects are found for the individual treatment. However, girls significantly increase their metacognitive skills and are more motivated than boys, when using an individual treatment. Additionally, a mediation analysis shows that motivation partially mediates the cooperative treatment and metacognitive skills. Based on these results, we recommend a combination of peer discussions and teacher feedback in physics courses.

[1]  Elizabeth Tricomi,et al.  Effects of intrinsic motivation on feedback processing during learning , 2015, NeuroImage.

[2]  Kellah M. Edens.  The Interaction of Pedagogical Approach, Gender, Self-Regulation, and Goal Orientation Using Student Response System Technology , 2008 .

[3]  A. Desoete.  Multi-method assessment of metacognitive skills in elementary school children: how you test is what you get , 2008 .

[4]  Anique B. H. de Bruin,et al.  Monitoring and regulation of learning in medical education: the need for predictive cues , 2017, Medical education.

[5]  C. Carvalho,et al.  Teachers’ Feedback: Exploring Differences in Students’ Perceptions , 2014 .

[6]  A. Hofman,et al.  The impact of interactive engagement methods on students' academic achievement , 2014 .

[7]  Lorena Blasco-Arcas,et al.  Using clickers in class. The role of interactivity, active collaborative learning and engagement in learning performance , 2013, Comput. Educ..

[8]  Angel Hoekstra,et al.  How clicker use facilitates existing pedagogical practices in higher education: data from interdisciplinary research on student response systems , 2012 .

[9]  J. Hattie,et al.  The Power of Feedback , 2007 .

[10]  Aime A. Levesque,et al.  Using Clickers to Facilitate Development of Problem-Solving Skills , 2011, CBE life sciences education.

[11]  Carol Sansone,et al.  Interest as the Missing Motivator in Self-Regulation , 2005 .

[12]  Daniel B. King,et al.  Gender Differences in the Use and Effectiveness of Personal Response Devices , 2008 .

[13]  Chandralekha Singh,et al.  Assessing Student Expertise in Introductory Physics with Isomorphic Problems. II. Effect of Some Potential Factors on Problem Solving and Transfer. , 2008, 1602.07678.

[14]  Gregory Schraw,et al.  Metacognitive Knowledge and Field-based Science Learning in an Outdoor Environmental Education Program , 2012 .

[15]  Jane E Caldwell,et al.  Clickers in the large classroom: current research and best-practice tips. , 2007, CBE life sciences education.

[16]  Chun-Yen Chang,et al.  Do we click in the right way? A meta-analytic review of clicker-integrated instruction , 2016 .

[17]  Jennifer K. Knight,et al.  Cues Matter: Learning Assistants Influence Introductory Biology Student Interactions during Clicker-Question Discussions , 2015, CBE life sciences education.

[18]  W. Groot,et al.  Toward Reducing Anxiety and Increasing Performance in Physics Education: Evidence from a Randomized Experiment , 2019, Research in Science Education.

[19]  Alison Crowe,et al.  Biology in bloom: implementing Bloom's Taxonomy to enhance student learning in biology. , 2008, CBE life sciences education.

[20]  Michael N. Mitchell.  Stata for the Behavioral Sciences , 2015 .

[21]  Kari Smith,et al.  Formative Assessment and Feedback: Making Learning Visible. , 2012 .

[22]  Josip Slisko,et al.  Software Socrative and Smartphones as Tools For Implementation of Basic Processes of Active Physics Learning in Classroom: An Initial Feasibility Study With Prospective Teachers , 2013 .

[23]  MacKenzie R. Stetzer,et al.  A Campus-Wide Investigation of Clicker Implementation: The Status of Peer Discussion in STEM Classes , 2016, CBE life sciences education.

[24]  S. DiCarlo,et al.  Peer instruction enhanced meaningful learning: ability to solve novel problems. , 2005, Advances in physiology education.

[25]  Scott Freeman,et al.  Is It the Intervention or the Students? Using Linear Regression to Control for Student Characteristics in Undergraduate STEM Education Research , 2014, CBE life sciences education.

[26]  J. Bois,et al.  Teacher feedback and interactions in physical education: Effects of student gender and physical activities , 2007 .

[27]  Kevin C. Almeroth,et al.  Clickers in college classrooms: Fostering learning with questioning methods in large lecture classes , 2009 .

[28]  Isabel Buil,et al.  Do clickers enhance learning? A control-value theory approach , 2016, Comput. Educ..

[29]  Alf Inge Wang,et al.  The wear out effect of a game-based student response system , 2015, Comput. Educ..

[30]  Rune Johan Krumsvik,et al.  Creating formative feedback spaces in large lectures , 2015, Comput. Educ..

[31]  Amy M. Shapiro,et al.  Clickers can promote fact retention but impede conceptual understanding: The effect of the interaction between clicker use and pedagogy on learning , 2017, Comput. Educ..

[32]  J. Oigara,et al.  Teaching and Learning with Clickers in Higher Education. , 2014 .

[33]  P. Paoloni,et al.  Hope and anxiety in physics class: Exploring their motivational antecedents and influence on metacognition and performance , 2017 .

[34]  M. Sobel.  Asymptotic Confidence Intervals for Indirect Effects in Structural Equation Models , 1982 .

[35]  E. Mazur,et al.  Peer Instruction: Ten years of experience and results , 2001 .

[36]  Catherine H. Crouch,et al.  Reducing the gender gap in the physics classroom , 2006 .

[37]  Clyde Freeman Herreid,et al.  Gender differences in student performance in large lecture classrooms using personal response systems (‘clickers’) with narrative case studies , 2012 .

[38]  Gregory P. Thomas,et al.  Changing the Metacognitive Orientation of a Classroom Environment to Stimulate Metacognitive Reflection Regarding the Nature of Physics Learning , 2013 .

[39]  Michael E. Lantz,et al.  Effectiveness of clickers: Effect of feedback and the timing of questions on learning , 2014, Comput. Hum. Behav..

[40]  Jeffrey M. Wooldridge,et al.  Solutions Manual and Supplementary Materials for Econometric Analysis of Cross Section and Panel Data , 2003 .

[41]  J. Flavell.  Metacognition and Cognitive Monitoring: A New Area of Cognitive-Developmental Inquiry. , 1979 .

[42]  Philip H. Winne,et al.  Studying as self-regulated learning. , 1998 .

[43]  Shawn M. Glynn,et al.  Science Motivation Questionnaire: Construct Validation With Nonscience Majors , 2009 .

[44]  Leonard R. Bachman,et al.  A study of classroom response system clickers: Increasing student engagement and performance in a large undergraduate lecture class on architectural research , 2011 .

[45]  William B Wood,et al.  Teaching more by lecturing less. , 2005, Cell biology education.

[46]  R. Pekrun.  The Control-Value Theory of Achievement Emotions: Assumptions, Corollaries, and Implications for Educational Research and Practice , 2006 .

[47]  María-del-Mar Camacho-Miñano,et al.  Useful interactive teaching tool for learning: clickers in higher education† , 2016, Interact. Learn. Environ..

[48]  Kathryn E. Perez,et al.  Does Displaying the Class Results Affect Student Discussion during Peer Instruction? , 2010, CBE life sciences education.

[49]  Gregory Schraw,et al.  Issues in the measurement of metacognition , 2000 .

[50]  Robin Kay,et al.  Examining the Use of Audience Response Systems in Secondary School Classrooms: A Formative Analysis. , 2010 .

[51]  J. C. Gagne.  The impact of clickers in nursing education: a review of literature. , 2011 .

[52]  Zhonggen Yu,et al.  The Impact of Clickers Instruction on Cognitive Loads and Listening and Speaking Skills in College English Class , 2014, PloS one.

[53]  Chang Liu,et al.  The Influence of Clickers Use on Metacognition and Learning Outcomes in College English Classroom , 2014, Int. J. Inf. Commun. Technol. Educ..

[54]  Michelle K. Smith,et al.  Why Peer Discussion Improves Student Performance on In-Class Concept Questions , 2009, Science.

[55]  Julia Kokina,et al.  Using Socrative to Enhance Instruction in an Accounting Classroom , 2017 .

[56]  J. Stowell,et al.  Benefits of Electronic Audience Response Systems on Student Participation, Learning, and Emotion , 2007 .

[57]  S. A. Coutinho,et al.  The Relationship between the Need for Cognition, Metacognition, and Intellectual Task Performance. , 2006 .

[58]  C. Evans.  Making Sense of Assessment Feedback in Higher Education , 2013 .

[59]  D. A. Kenny,et al.  The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. , 1986, Journal of personality and social psychology.

[60]  J. Kirby,et al.  Approaches to learning, need for cognition, and strategic flexibility among university students. , 2003, The British journal of educational psychology.

[61]  Marianne Fallon,et al.  High-Tech Versus Low-Tech Instructional Strategies , 2011 .

[62]  Patrick Onghena,et al.  A comparative study of peer and teacher feedback and of various peer feedback forms in a secondary school writing curriculum , 2010 .

[63]  Joshua D. Angrist,et al.  Mostly Harmless Econometrics: An Empiricist's Companion , 2008 .

[64]  K. Krauter,et al.  Combining Peer Discussion with Instructor Explanation Increases Student Learning from In-Class Concept Questions , 2011, CBE life sciences education.

[65]  P. Pintrich,et al.  Motivational and self-regulated learning components of classroom academic performance. , 1990 .

[66]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[67]  E. Deci,et al.  Need satisfaction and the self-regulation of learning , 1996 .

[68]  M. Birenbaum.  Assessment 2000: Towards a Pluralistic Approach to Assessment , 1996 .

[69]  Robin H. Kay,et al.  Examining the benefits and challenges of using audience response systems: A review of the literature , 2009, Comput. Educ..

[70]  Kimberly D Tanner,et al.  Talking to learn: why biology students should be talking in classrooms and how to make it happen. , 2009, CBE life sciences education.

[71]  Pavlo D. Antonenko,et al.  The impact of collaborative and individualized student response system strategies on learner motivation, metacognition, and knowledge transfer , 2012, J. Comput. Assist. Learn..

[72]  Nathaniel J. Hunsu,et al.  A meta-analysis of the effects of audience response systems (clicker-based technologies) on cognition and affect , 2016, Comput. Educ..

[73]  Felicia M. Moore,et al.  Girls and science: A review of four themes in the science education literature , 2008 .

[74]  R. Murnane,et al.  Methods Matter: Improving Causal Inference in Educational and Social Science Research , 2010 .

[75]  Sarah K. Tauber,et al.  Beliefs about memory decline in aging do not impact judgments of learning (JOLs): A challenge for belief-based explanations of JOLs , 2019, Memory & cognition.

[76]  Jerry Chih-Yuan Sun,et al.  Application of a Gamified Interactive Response System to Enhance the Intrinsic and Extrinsic Motivation, Student Engagement, and Attention of English Learners , 2018, J. Educ. Technol. Soc..

[77]  Carla Haelermans,et al.  The effect of individualized digital practice at home on math skills-Evidence from a two-stage experiment on whether and why it works , 2017, Comput. Educ..

[78]  Katherine A. Rawson,et al.  Metacognitive monitoring during criterion learning: When and why are judgments accurate? , 2014, Memory & cognition.

[79]  P. Black,et al.  Assessment and Classroom Learning , 1998 .

[80]  P. Winne,et al.  Feedback and Self-Regulated Learning: A Theoretical Synthesis , 1995 .

[81]  Brent R. Moulton.  Random group effects and the precision of regression estimates , 1986 .

[82]  Nuri Balta,et al.  Using Socrative software for instant formative feedback in physics courses , 2018, Education and Information Technologies.

[83]  Michele H. Jackson,et al.  The learning environment in clicker classrooms: student processes of learning and involvement in large university‐level courses using student response systems , 2007 .

[84]  Reinhard Pekrun,et al.  Boredom in achievement settings: Exploring control–value antecedents and performance outcomes of a neglected emotion. , 2010 .

[85]  Helena Seli,et al.  Metacognition and the influence of polling systems: how do clickers compare with low technology systems , 2013, Educational Technology Research and Development.

[86]  Beth Simon,et al.  Peer instruction: do students really learn from peer discussion in computing? , 2011, ICER.

[87]  MacKenzie R. Stetzer,et al.  What are Middle School Students Talking About During Clicker Questions? Characterizing Small-Group Conversations Mediated by Classroom Response Systems , 2015, Journal of Science Education and Technology.

[88]  E. Deci,et al.  Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. , 2000, The American psychologist.

[89]  Katelyn M. Southard,et al.  Understanding Clicker Discussions: Student Reasoning and the Impact of Instructional Cues , 2013, CBE life sciences education.

[90]  V. Shute.  Focus on Formative Feedback , 2008 .

[91]  B. Zimmerman,et al.  Self-regulation of learning: Process approaches to personal development. , 2012 .

[92]  Y. Dori,et al.  Students’ Metacognition and Metacognitive Strategies in Science Education , 2018 .

[93]  Lucy Tiplady,et al.  Designing for Formative Assessment: A Toolkit for Teachers , 2018 .

[94]  Gregory Schraw,et al.  Assessing metacognitive awareness , 1994 .

[95]  Daniel Zingaro,et al.  Peer Instruction in computing: The value of instructor intervention , 2014, Comput. Educ..

[96]  Robert A. Bartsch,et al.  Examining the Effects of an Electronic Classroom Response System on Student Engagement and Performance , 2011 .

[97]  M. Tavakol,et al.  Making sense of Cronbach's alpha , 2011, International journal of medical education.

[98]  Mahmoud A Kaddoura,et al.  Think Pair Share: A Teaching Learning Strategy to Enhance Students' Critical Thinking. , 2013 .