Written justifications to multiple-choice concept questions during active learning in class

ABSTRACT Increasingly, instructors of large, introductory STEM courses are having students actively engage during class by answering multiple-choice concept questions individually and in groups. This study investigates the use of a technology-based tool that allows students to answer such questions during class. The tool also allows the instructor to prompt students to provide written responses to justify the selection of the multiple-choice answer that they have chosen. We hypothesize that prompting students to explain and elaborate on their answer choices leads to greater focus and use of normative scientific reasoning processes, and will allow them to answer questions correctly more often. The study contains two parts. First, a crossover quasi-experimental design is employed to determine the influence of asking students to individually provide written explanations (treatment condition) of their answer choices to 39 concept questions as compared to students who do not. Second, we analyze a subset of the questions to see whether students identify the salient concepts and use appropriate reasoning in their explanations. Results show that soliciting written explanations can have a significant influence on answer choice and, when it does, that influence is usually positive. However, students are not always able to articulate the correct reason for their answer.

[1]  Barbara M. Olds,et al.  How to Create a Concept Inventory: The Thermal and Transport Concept Inventory , 2007 .

[2]  Robin H. Kay,et al.  Examining the benefits and challenges of using audience response systems: A review of the literature , 2009, Comput. Educ..

[3]  Jeffrey E. Froyd,et al.  Fidelity of Implementation of Research‐Based Instructional Strategies (RBIS) in Engineering Science Courses , 2013 .

[4]  Shannon D. Willoughby,et al.  Technology talks: Clickers and grading incentive in the large lecture hall , 2009 .

[5]  D. Flemons Thinking on paper , 1998 .

[6]  Vicente A Talanquer,et al.  Chemistry Education: Ten Heuristics To Tame , 2014 .

[7]  Lillian C. McDermott,et al.  Physics education research: the key to student learning , 2001 .

[8]  Loretta L. Jones,et al.  A Review of Literature Reports of Clickers Applicable to College Chemistry Classrooms , 2008 .

[9]  Andrew Elby,et al.  Resources , framing , and transfer , 2004 .

[10]  Arthur N. Applebee Writing and Reasoning , 1984 .

[11]  Mark Urban-Lurain,et al.  Using computerized lexical analysis of student writing to support Just-in-Time teaching in large enrollment STEM courses , 2013, 2013 IEEE Frontiers in Education Conference (FIE).

[12]  Andrew F. Heckler,et al.  Patterns of Response Times and Response Choices to Science Questions: The Influence of Relative Processing Time , 2015, Cogn. Sci..

[13]  Mark C. James The effect of grading incentive on student discourse in Peer Instruction , 2006 .

[14]  Charles Xie,et al.  A visual approach to nanotechnology education , 2012 .

[15]  Elizabeth S. Charles,et al.  When Talking Is Better Than Staying Quiet , 2009 .

[16]  Chandralekha Singh,et al.  Impact of peer interaction on conceptual test performance , 2005, 1602.07661.

[17]  Deborah Allen,et al.  Approaches to biology teaching and learning: understanding the wrong answers--teaching toward conceptual change. , 2005, Cell biology education.

[18]  W. L. Yarroch The implications of content versus item validity on science tests , 1991 .

[19]  Bill Jay Brooks,et al.  Student response times to in-class thermodynamics concept questions: a window into students’ thinking processes , 2016 .

[20]  Robert J. Tierney,et al.  The Effects of Reading and Writing upon Thinking Critically. , 1989 .

[21]  W. Wood Clickers: A Teaching Gimmick that Works , 2004 .

[22]  E. Mazur,et al.  Peer Instruction: Ten years of experience and results , 2001 .

[23]  Ann L. Brown,et al.  How people learn: Brain, mind, experience, and school. , 1999 .

[24]  Eric Mazur,et al.  Peer Instruction: A User's Manual , 1996 .

[25]  Michelene T. H. Chi,et al.  Eliciting Self-Explanations Improves Understanding , 1994, Cogn. Sci..

[26]  Richard Braddock,et al.  Research in Written Composition , 2012 .

[27]  Debra L. Linton,et al.  Identifying Key Features of Effective Active Learning: The Effects of Writing and Peer Discussion , 2014, CBE life sciences education.

[28]  Lillian C. McDermott,et al.  Oersted Medal Lecture 2001: “Physics Education Research—The Key to Student Learning” , 2001 .

[29]  Paul J. Feltovich,et al.  Categorization and Representation of Physics Problems by Experts and Novices , 1981, Cogn. Sci..

[30]  Pinchas Tamir,et al.  Training Teachers to Teach Effectively in the Laboratory. , 1989 .

[31]  Pinchas Tamir,et al.  Justifying the selection of answers in multiple choice items , 1990 .

[32]  Zuzana Haláková,et al.  Two Kinds of Conceptual Problems in Chemistry Teaching , 2007 .

[33]  Anne Ruggles Gere,et al.  Talking About Writing: The Language of Writing Groups. , 1985 .

[34]  Katharine D. Owens,et al.  Using Conceptests to Assess and Improve Student Conceptual Understanding in Introductory Geoscience Courses , 2006 .

[35]  H. Mandl,et al.  Learning from Worked-Out Examples: The Effects of Example Variability and Elicited Self-Explanations , 1998, Contemporary educational psychology.

[36]  Ken A Paller,et al.  What makes recognition without awareness appear to be elusive? Strategic factors that influence the accuracy of guesses. , 2010, Learning & memory.

[37]  Susan R. Singer,et al.  Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering. , 2012 .

[38]  Ian D. Beatty,et al.  Designing effective questions for classroom response system teaching , 2005, physics/0508114.

[39]  B. Rittle-Johnson,et al.  Conceptual and procedural knowledge of mathematics: Does one lead to the other? , 1999 .

[40]  Paul S. Steif,et al.  Student Thinking about Static Equilibrium: Insights from Written Explanations to a Concept Question , 2008 .

[41]  Kjetil L. Nielsen,et al.  How the initial thinking period affects student argumentation during peer instruction: students’ experiences versus observations , 2016 .

[42]  David C. Haak,et al.  Increased Structure and Active Learning Reduce the Achievement Gap in Introductory Biology , 2011, Science.

[43]  Michelle K. Smith,et al.  Active learning increases student performance in science, engineering, and mathematics , 2014, Proceedings of the National Academy of Sciences.

[44]  Douglas K. Duncan,et al.  Clickers: A New Teaching Aid with Exceptional Promise , 2006 .

[45]  Kurt VanLehn,et al.  Development of Professional Expertise: Acquiring Conceptual Expertise from Modeling: The Case of Elementary Physics , 2009 .

[46]  Alexander Renkl,et al.  Different roads lead to Rome: the case of principle-based cognitive skills , 2015 .

[47]  Michelle K. Smith,et al.  Why Peer Discussion Improves Student Performance on In-Class Concept Questions , 2009, Science.

[48]  Evelyn S. Johnson,et al.  Fidelity of Implementation , 2021, Encyclopedia of Autism Spectrum Disorders.

[49]  L. Rivard,et al.  A review of writing to learn in science: Implications for practice and research , 1994 .

[50]  Adam R. Lucas Using Peer Instruction and I-Clickers to Enhance Student Participation in Calculus , 2009 .

[51]  Julie A. Bianchini,et al.  Expanding underrepresented minority participation: America's science and technology talent at the crossroads , 2013 .

[52]  David F. Treagust,et al.  The development of a two-tier multiple-choice diagnostic instrument for evaluating secondary school students’ ability to describe and explain chemical reactions using multiple levels of representation , 2007 .

[53]  Brian Hand,et al.  A secondary reanalysis of student perceptions of non‐traditional writing tasks over a ten year period , 2009 .

[54]  Louis Deslauriers,et al.  Improved Learning in a Large-Enrollment Physics Class , 2011, Science.

[55]  K. Krauter,et al.  Combining Peer Discussion with Instructor Explanation Increases Student Learning from In-Class Concept Questions , 2011, CBE life sciences education.

[56]  Patricia M. Rowell Learning in School Science: The Promises and Practices of Writing. , 1997 .

[57]  Nicole Graulich Intuitive Judgments Govern Students' Answering Patterns in Multiple-Choice Exercises in Organic Chemistry , 2015 .

[58]  L. V. Dijk,et al.  Interactive lectures in engineering education , 2001 .

[59]  Lee Odell,et al.  The Process of Writing and the Process of Learning. , 1980 .

[60]  M. Koretsky,et al.  Querying the Questions: Student Responses and Reasoning in an Active Learning Class , 2016 .

[61]  Jane E Caldwell,et al.  Clickers in the large classroom: current research and best-practice tips. , 2007, CBE life sciences education.

[62]  R. Hake Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses , 1998 .

[63]  Georgios Tsaparlis,et al.  Conceptual versus Algorithmic Learning in High School Chemistry: The Case of Basic Quantum Chemical Concepts--Part 1. Statistical Analysis of a Quantitative Study. , 2008 .

[64]  Naola VanOrden Is writing an effective way to learn chemical concepts? Classroom-based research , 1990 .

[65]  Naola VanOrden,et al.  Critical-thinking writing assignments in general chemistry , 1987 .

[66]  Perry D. Klein Reopening Inquiry into Cognitive Processes in Writing-To-Learn , 1999 .

[67]  William B Wood,et al.  Teaching more by lecturing less. , 2005, Cell biology education.

[68]  Susan C. Nurrenbern,et al.  Conceptual Questions and Challenge Problems , 1998 .

[69]  Bill Jay Brooks,et al.  The AIChE "Concept Warehouse": A Web-Based Tool to Promote Concept-Based Instruction. , 2014 .

[70]  D. Kahneman A perspective on judgment and choice: mapping bounded rationality. , 2003, The American psychologist.

[71]  Mark Urban-Lurain,et al.  What Are They Thinking? Automated Analysis of Student Writing about Acid–Base Chemistry in Introductory Biology , 2012, CBE life sciences education.

[72]  Matthew Pilarz,et al.  Research-Based Implementation of Peer Instruction: A Literature Review , 2015, CBE life sciences education.

[73]  B. Rittle-Johnson,et al.  Conceptual and Procedural Knowledge of Mathematics : Does One Lead to the Other ? , 2004 .