A Pilot Meta-Analysis of Computer-Based Scaffolding in STEM Education

This paper employs meta-analysis to determine the influence of computer-based scaffolding characteristics and study and test score quality on cognitive outcomes in science, technology, engineering, and mathematics education at the secondary, college, graduate, and adult levels. Results indicate that (a) computer-based scaffolding positively influences learning (g = 0.53), (b) studies with zero threats to internal validity had lower effect sizes than studies with two threats, (c) studies with one threat to external validity had higher effect sizes than studies with zero threats, (d) studies with no fading had higher effect sizes than studies with fixed fading, and (e) students performed better when using conceptual scaffolds than with metacognitive scaffolds. There were no differences based on study design, generic vs. specific, paired intervention, assessment level, or intended learning outcome. Meta-regression indicated that fading or lack thereof explained 30% of the variability in outcomes. The significance of this paper lies in its potential to steer scaffold designers away from fixed fading and metacognitive scaffolds, and toward studying scaffolding in authentic contexts, rather than laboratories. Furthermore, this study indicates that a more comprehensive scaffolding meta-analysis is warranted.

[1]  L. Hedges,et al.  The Handbook of Research Synthesis and Meta-Analysis , 2009 .

[2]  Brian R. Belland,et al.  Distributed Cognition as a Lens to Understand the Effects of Scaffolds: The Role of Transfer of Responsibility , 2011 .

[3]  Talbot County,et al.  The Common Core State Standards , 2013 .

[4]  Marcia C. Linn,et al.  Designing the Knowledge Integration Environment , 2000 .

[5]  Susan M. Land,et al.  Scaffolding Peer-questioning Strategies to Facilitate Metacognition During Online Small Group Discussion , 2005 .

[6]  R. Mayer.  Systematic Thinking Fostered by Illustrations in Scientific Text , 1989 .

[7]  Susan M. Land,et al.  A conceptual framework for scaffolding III-structured problem-solving processes using question prompts and peer interactions , 2004 .

[8]  Susan M. Land,et al.  Scaffolding students’ problem-solving processes in an ill-structured task using question prompts and peer interactions , 2003 .

[9]  E. Davis.  Prompting Middle School Science Students for Productive Reflection: Generic and Directed Prompts , 2003 .

[10]  Joseph Krajcik,et al.  Synergy Between Teacher Practices and Curricular Scaffolds to Support Students in Using Domain-Specific and Domain-General Knowledge in Writing Arguments to Explain Phenomena , 2009 .

[11]  Meredith D. Gall,et al.  Educational Research: An Introduction , 1965 .

[12]  S. Puntambekar,et al.  Tools for Scaffolding Students in a Complex Learning Environment: What Have We Gained and What Have We Missed? , 2005 .

[13]  Joseph Krajcik,et al.  The design of guided learner-adaptable scaffolding in interactive learning environments , 1996, CHI.

[14]  G. Glass.  Primary, Secondary, and Meta-Analysis of Research1 , 1976 .

[15]  H. Lee Swanson,et al.  A Selective Synthesis of the Experimental Literature on Dynamic Assessment , 2001 .

[16]  Brian J. Reiser,et al.  Scaffolding Complex Learning: The Mechanisms of Structuring and Problematizing Student Work , 2004, The Journal of the Learning Sciences.

[17]  Timothy Shanahan,et al.  Common Core State Standards: A New Role for Writing. , 2015 .

[18]  H. Swanson,et al.  Instructing Adolescents with Learning Disabilities , 2003, Journal of learning disabilities.

[19]  B. Belland.  Scaffolding: Definition, Current Debates, and Future Directions , 2014 .

[20]  C. Reigeluth.  Technology-Enhanced Support Strategies for Inquiry Learning , 2008 .

[21]  Dean P. Foster,et al.  The risk inflation criterion for multiple regression , 1994 .

[22]  S. Urbina,et al.  Psychological testing, 7th ed. , 1997 .

[23]  John G. Hedberg,et al.  Instructional design theories and models volume II , 2000 .

[24]  R. Linn.  Educational measurement, 3rd ed. , 1989 .

[25]  Robert M. Bernard,et al.  Instructional Interventions Affecting Critical Thinking Skills and Dispositions: A Stage 1 Meta-Analysis , 2008 .

[26]  David H. Jonassen,et al.  Handbook of Research for educational Communications and Technology , 1997 .

[27]  Mitchell J. Nathan,et al.  A theory of algebra-word-problem comprehension and its implications for the design of learning environments. , 1992 .

[28]  Brenda Sugrue,et al.  A Theory‐Based Framework for Assessing Domainl‐Specific Problem‐Solving Ability , 2005 .

[29]  Thomas P. Hogan,et al.  An Empirical Study of Reporting Practices Concerning Measurement Validity , 2004 .

[30]  Kevin Oliver,et al.  Student Management of Web-Based Hypermedia Resources During Open-Ended Problem Solving , 2000 .

[31]  Miky Ronen,et al.  Simulation - a bridge between theory and reality: the case of electric circuits , 2001, J. Comput. Assist. Learn..

[32]  Susanne P. Lajoie,et al.  Constructing knowledge in the context of BioWorld , 2001 .

[33]  J. Kolodner,et al.  Toward implementing distributed scaffolding: Helping students learn science from design , 2005 .

[34]  Rich Feller,et al.  Aligning School Counseling, the Changing Workplace, and Career Development Assumptions. , 2003 .

[35]  L. Vygotsky,et al.  Thought and Language , 1963 .

[36]  E. Chambers.  An Introduction to Meta-Analysis With Articles From The Journal of Educational Research (1992-2002) , 2004 .

[37]  Yu-Chun Kuo,et al.  A Meta-Analysis of Problem-Based Learning Corrected for Attenuation, and Accounting for Internal Threats , 2010 .

[38]  Harris Cooper,et al.  Integrating research: A guide for literature reviews, 2nd ed. , 1989 .

[39]  R. D. Nordgren,et al.  Globalization and Education: What Students Will Need to Know and Be Able to Do in the Global Village , 2002 .

[40]  N. Tallent.  Psychological testing. , 1960, The American journal of nursing.

[41]  Brian J. Foley.  Visualization tools: Models, representations and knowledge integration , 1999 .

[42]  Harris Cooper,et al.  Integrating Research: A Guide for Literature Reviews , 1989 .

[43]  Krista Glazewski,et al.  A scaffolding framework to support the construction of evidence-based arguments among middle school students , 2008 .

[44]  D. Jonassen,et al.  Arguing to learn and learning to argue: design justifications and guidelines , 2010 .

[45]  Francis M. Dwyer,et al.  Effectiveness of Various Enhancement Strategies to Complement Animated Instruction: A Meta-Analytic Assessment , 2006 .

[46]  Roy,et al.  The Social and Technological Dimensions of Scaffolding and Related Theoretical Concepts for Learning , Education , and Human Activity , 2004 .

[47]  Paul Adams,et al.  Next Generation Science Standards , 2013 .

[48]  Cher Ping Lim,et al.  Scaffolding online historical inquiry tasks: A case study of two secondary school classrooms , 2008, Comput. Educ..

[49]  John J. Clement,et al.  Using Bridging Analogies and Anchoring Institutions to Seal with Students' Preconceptions in Physics , 1993 .

[50]  Bat-Sheva Eylon,et al.  From problem solving to a knowledge structure: An example from the domain of electromagnetism , 1997 .

[51]  Gary W. Ritter,et al.  Next Generation Science Standards , 2013 .

[52]  Brian R. Belland,et al.  Portraits of middle school students constructing evidence-based arguments during problem-based learning: the impact of computer-based scaffolds , 2010 .

[53]  P. Sleight,et al.  Publication bias , 1991, The Lancet.

[54]  M. Volman,et al.  Scaffolding in Teacher–Student Interaction: A Decade of Research , 2010 .

[55]  T. Anderson,et al.  Design-Based Research , 2012 .

[56]  Joseph Krajcik,et al.  A Scaffolding Design Framework for Software to Support Science Inquiry , 2004, The Journal of the Learning Sciences.

[57]  K. VanLehn.  The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems , 2011 .

[58]  Hossam Hamdy,et al.  BEME systematic review: Predictive values of measurements obtained in medical schools and future performance in medical practice , 2006, Medical teacher.

[59]  T. James,et al.  THE COUNCIL OF CHIEF STATE SCHOOL OFFICERS , 2009 .

[60]  J. Frederiksen,et al.  Inquiry, Modeling, and Metacognition: Making Science Accessible to All Students , 1998 .

[61]  M. S. Patel,et al.  An introduction to meta-analysis. , 1989, Health Policy.

[62]  Susan E. Newman,et al.  Cognitive Apprenticeship: Teaching the Craft of Reading, Writing, and Mathematics. Technical Report No. 403. , 1987 .

[63]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[64]  J. Loevinger,et al.  The attenuation paradox in test theory. , 1954, Psychological bulletin.

[65]  Pamela A. Moss,et al.  Standards for Reporting on Empirical Social Science Research in AERA Publications American Educational Research Association , 2006 .

[66]  Peg A Ertmer,et al.  Open Learning Environments, Foundations, Methods, and Models , 2012 .

[67]  L. Resnick,et al.  Knowing, Learning, and Instruction , 2018 .

[68]  J. Bruner,et al.  The role of tutoring in problem solving. , 1976, Journal of child psychology and psychiatry, and allied disciplines.

[69]  Peggy A. Ertmer,et al.  Validity and Problem-Based Learning Research: A Review of Instruments Used to Assess Intended Learning Outcomes , 2016 .

[70]  D. Laplane.  Thought and language. , 1992, Behavioural neurology.

[71]  Thomas Brush,et al.  Scaffolding critical reasoning about history and social issues in multimedia-supported learning environments , 2002 .