The Testing Effect: An Intervention on Behalf of Low-Skilled Comprehenders in General Chemistry

Past work has demonstrated that language comprehension ability correlates with general chemistry course performance with medium effect sizes. We demonstrate here that language comprehension’s strong cognitive grounding can be used to inform effective and equitable pedagogies, namely, instructional interventions that differentially aid low-skilled language comprehenders. We report the design, implementation, and assessment of such an intervention strategy. Guided by two models of comprehension, we predicted that a multiple pre-testing strategy would differentially aid low-skilled comprehenders in a general chemistry class. We also explored the effect of two question types (multiple choice and elaborative interrogation) on this intervention strategy. A within-subjects, learning-goals driven design was used to build the intervention into two semesters of the course; data generated by this approach were analyzed with hierarchical linear models. We found that the achievement gap between low- and high-skilled c...

[1]  Sachel M. Villafañe,et al.  Exploring diverse students' trends in chemistry self-efficacy throughout a semester of college-level preparatory chemistry , 2014 .

[2]  J. Fleiss Measuring nominal scale agreement among many raters. , 1971 .

[3]  Vicki Roth,et al.  The Workshop Chemistry Project: Peer-Led Team Learning , 1998 .

[4]  M. McDaniel,et al.  Test-enhanced learning in the classroom: long-term improvements from quizzing. , 2011, Journal of experimental psychology. Applied.

[5]  Loretta L. Jones,et al.  A Review of Literature Reports of Clickers Applicable to College Chemistry Classrooms , 2008 .

[6]  M. McDaniel,et al.  Testing the testing effect in the classroom , 2007 .

[7]  Carl C. Wamser,et al.  Peer-Led Team Learning in Organic Chemistry: Effects on Student Performance, Success, and Persistence in the Course , 2006 .

[8]  A. King,et al.  Effects of Guided Cooperative Questioning on Children's Knowledge Construction. , 1993 .

[9]  M. McDaniel,et al.  Generation and precision of elaboration: Effects on intentional and incidental learning. , 1987 .

[10]  N G HANAWALT,et al.  The effect of recall upon recognition. , 1961, Journal of experimental psychology.

[11]  Rayne A. Sperling,et al.  A comparison of the effects of students’ expository text comprehension strategies , 2010 .

[12]  Daniel T. Pyburn,et al.  Applying the Multilevel Framework of Discourse Comprehension To Evaluate the Text Characteristics of General Chemistry Textbooks , 2014 .

[13]  W. Kintsch,et al.  Differential effects of study and test trials on long-term recognition and recall , 1971 .

[14]  Daniel T. Pyburn,et al.  Assessing the relation between language comprehension and performance in general chemistry , 2013 .

[15]  Jeffrey D. Karpicke,et al.  The Power of Testing Memory Basic Research and Implications for Educational Practice , 2006 .

[16]  D. McNamara,et al.  Strategies to read and learn: overcoming learning by consumption , 2010, Medical education.

[17]  Michelle K. Smith,et al.  Why Peer Discussion Improves Student Performance on In-Class Concept Questions , 2009, Science.

[18]  Lydia T. Tien,et al.  Implementation of a Peer-Led Team Learning Instructional Approach in an Undergraduate Organic Chemistry Course. , 2002 .

[19]  Scott E. Lewis,et al.  Predicting at-risk students in general chemistry: comparing formal thought to a general achievement measure , 2007 .

[20]  Arthur C. Graesser,et al.  Computational Analyses of Multilevel Discourse Comprehension , 2011, Top. Cogn. Sci..

[21]  S. Lewis,et al.  Departing from Lectures: An Evaluation of a Peer-Led Guided Inquiry Alternative , 2005 .

[22]  D. Treagust Development and use of diagnostic tests to evaluate students’ misconceptions in science , 1988 .

[23]  Stacy D. Brown,et al.  A Process-Oriented Guided Inquiry Approach to Teaching Medicinal Chemistry , 2010, American Journal of Pharmaceutical Education.

[24]  Richard S. Moog,et al.  A Guided-Inquiry General Chemistry Course , 1999 .

[25]  Mark A. McDaniel,et al.  The benefits of embedded question adjuncts for low and high structure builders. , 2007 .

[26]  Dorothy L. Gabel,et al.  Improving Teaching and Learning through Chemistry Education Research: A Look to the Future , 1999 .

[27]  Christiaan Hamaker The Effects of Adjunct Questions on Prose Learning , 1986 .

[28]  Bennet B. Murdock,et al.  Effects of Prior Free Recall Testing on Final Recall and Recognition. , 1971 .

[29]  Georgios Tsaparlis,et al.  Atomic and Molecular Structure in Chemical Education: A Critical Analysis from Various Perspectives of Science Education. , 1997 .

[30]  A. Glass,et al.  Combined online and in‐class pretesting improves exam performance in general psychology , 2008 .

[31]  S. Lewis,et al.  Seeking effectiveness and equity in a large college chemistry course: an HLM investigation of Peer-Led Guided Inquiry , 2008 .

[32]  Katherine L. McNeill,et al.  Learning‐goals‐driven design model: Developing curriculum materials that align with national standards and incorporate project‐based pedagogy , 2008 .

[33]  K. VanLehn,et al.  Scaffolding Deep Comprehension Strategies Through Point&Query, AutoTutor, and iSTART , 2005 .

[34]  Doug Rohrer,et al.  Tests enhance the transfer of learning. , 2010, Journal of experimental psychology. Learning, memory, and cognition.

[35]  M. Gernsbacher,et al.  Investigating differences in general comprehension skill. , 1990, Journal of experimental psychology. Learning, memory, and cognition.

[36]  Developmental differences in study behavior , 1999 .

[37]  John T. Guthrie,et al.  Interactions Among Elaborative Interrogation, Knowledge, and Interest in the Process of Constructing Knowledge From Text , 2004 .

[38]  David F. Treagust,et al.  Grade-12 students' misconceptions of covalent bonding and structure , 1989 .

[39]  Jennifer Loertscher,et al.  Lecture‐free biochemistry , 2007, Biochemistry and molecular biology education : a bimonthly publication of the International Union of Biochemistry and Molecular Biology.

[40]  Philippe C. Duchastel,et al.  Retention of prose following testing with different types of tests , 1981 .

[41]  A. H. Johnstone,et al.  You Can’t Get There from Here1 , 2010 .

[42]  Regina F. Frey,et al.  Peer-Led Team Learning in General Chemistry: Implementation and Evaluation , 2008 .

[43]  Marilyne Stains,et al.  Implementing Process-Oriented, Guided-Inquiry Learning for the First Time: Adaptations and Short-Term Impacts on Students' Attitude and Performance. , 2013 .

[44]  Stacey Lowery Bretz,et al.  Development of the Bonding Representations Inventory to Identify Student Misconceptions about Covalent and Ionic Bonding Representations. , 2014 .

[45]  Stacey Lowery Bretz,et al.  Development of the enzyme–substrate interactions concept inventory , 2012, Biochemistry and molecular biology education : a bimonthly publication of the International Union of Biochemistry and Molecular Biology.

[46]  Mark A. McDaniel,et al.  Test-Enhanced Learning in a Middle School Science Classroom: The Effects of Quiz Frequency and Placement. , 2011 .

[47]  Mark A. McDaniel,et al.  The Limited Benefits of Rereading Educational Texts. , 2009 .

[48]  Albertine Krohn,et al.  An evaluation of the Toledo chemistry placement examination , 1963 .

[49]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[50]  Samuel Pazicni,et al.  Characterizing illusions of competence in introductory chemistry students , 2014 .

[51]  J. Fincham,et al.  Financial Realities Affect Political Support for Health Care Reform , 2010, American Journal of Pharmaceutical Education.

[52]  M. R. Rubin,et al.  Chem-2-Chem: A One-to-One Supportive Learning Environment for Chemistry , 2005 .

[53]  Mark A. McDaniel,et al.  Using quizzes to enhance summative-assessment performance in a web-based class: An experimental study , 2012 .

[54]  Diane M. Bunce,et al.  Comparing the Effectiveness on Student Achievement of a Student Response System versus Online WebCT Quizzes , 2006 .

[55]  Mark A. McDaniel,et al.  Altering memory representations through retrieval. , 1985 .

[56]  William G. Holliday,et al.  Students' comprehension of science textbooks using a question-based reading strategy , 2010 .

[57]  S. Lewis,et al.  Evaluating Peer-Led Team Learning across the two semester General Chemistry sequence , 2012 .

[58]  Kevin C. Almeroth,et al.  Clickers in college classrooms: Fostering learning with questioning methods in large lecture classes , 2009 .

[59]  A. Johnstone Chemistry Teaching - Science or Alchemy? 1996 Brasted Lecture , 1997 .

[60]  M. McDaniel,et al.  Altering memory through recall: The effects of cue-guided retrieval processing , 1989, Memory & cognition.

[61]  Mark A. McDaniel,et al.  Learning With Analogy and Elaborative Interrogation , 1996 .

[62]  C. Bereiter,et al.  Use of Thinking Aloud in Identification and Teaching of Reading Comprehension Strategies , 1985 .

[63]  Henry L. Roediger,et al.  Test format and corrective feedback modify the effect of testing on long-term retention , 2007 .

[64]  P. Pintrich,et al.  Motivational and self-regulated learning components of classroom academic performance. , 1990 .

[65]  T. Willoughby,et al.  Evaluating the Efficacy of Elaborative Strategies for Remembering Expository Text , 1999 .

[66]  Christopher F. Bauer,et al.  Attitude toward Chemistry: A Semantic Differential Instrument for Assessing Curriculum Impacts , 2008 .

[67]  Ann L. Brown,et al.  Reciprocal Teaching of Comprehension-Fostering and Comprehension-Monitoring Activities , 1984 .

[68]  Henry L. Roediger,et al.  Examining the Testing Effect with Open-and Closed-book Tests , 2022 .

[69]  Genna Angello,et al.  Multiple-Choice Tests Exonerated, at Least of Some Charges , 2012, Psychological science.

[70]  Patrick J. P. Brown Process-oriented guided-inquiry learning in an introductory anatomy and physiology course with a diverse student population. , 2010, Advances in physiology education.

[71]  Richard S. Moog,et al.  Process Oriented Guided Inquiry Learning (POGIL) , 2008 .

[72]  Scott E. Lewis,et al.  Retention and Reform: An Evaluation of Peer-Led Team Learning , 2011 .

[73]  Robert A. Bjork,et al.  Learning from tests: Effects of spacing , 1977 .