A meta-analysis of the effects of audience response systems (clicker-based technologies) on cognition and affect

Audience Response Systems (ARS) are thought to be a good way of using technology to increase engagement in the classroom and have been widely adopted by many instructors seeking to improve academic performance through student engagement. While researchers have examined the degree to which they promote cognitive and non-cognitive learning outcomes in the classroom, most of their findings are largely mixed and inconclusive. This meta-analysis seeks to resolve the conflicting findings. Specifically, the meta-analysis compared classrooms that did, and did not use ARS-based technologies on different cognitive and non-cognitive learning outcomes to examine the potential effects of using ARS. Overall, we found small but significant effects of using ARS-based technologies on a number of desirable cognitive and non-cognitive learning outcomes. Further analysis revealed that knowledge domain, class size, and the use of clicker questions, are among factors that significantly moderated the summary effect sizes observed among the studies in the meta-analysis. These findings hold significant implication for the implementation of clicker-based technologies in the classroom. Clickers have a small but significant effect on cognitive learning outcomes.Clickers have a near medium effect on non-cognitive learning outcomes.These effects are moderated by a number of moderators.Findings have implication for instructional design and research.

[1]  Forward Transfer of Different Reading Strategies Evoked by Testlike Events in Mathematics Text. , 1975 .

[2]  Robert M. Bernard,et al.  How Does Distance Education Compare With Classroom Instruction? A Meta-Analysis of the Empirical Literature , 2004 .

[3]  Robert S. Feldman,et al.  Promoting Active Learning Using Individual Response Technology in Large Introductory Psychology Classes , 2007 .

[4]  Jeffrey D. Karpicke,et al.  The Power of Testing Memory Basic Research and Implications for Educational Practice , 2006 .

[5]  Jeremy D. Finn,et al.  Tennessee's Class Size Study: Findings, Implications, Misconceptions , 1999 .

[6]  Ernest T. Pascarella,et al.  How College Affects Students: A Third Decade of Research. Volume 2. , 2005 .

[7]  M. Doucet,et al.  Effect of using an audience response system on learning environment, motivation and long-term retention, during case-discussions in a large group of undergraduate veterinary clinical pharmacology students , 2009, Medical teacher.

[8]  M. Oliver,et al.  Electronic Voting Systems for Lectures then and Now: A Comparison of Research and Practice , 2007 .

[9]  Mark W. Lipsey,et al.  Practical Meta-Analysis , 2000 .

[10]  Christine McDonald,et al.  CRiSP: An Instrument for Assessing Student Perceptions of Classroom Response Systems , 2015 .

[11]  Forward Transfer of Different Reading Strategies Evoked by Adjunct Questions in Science Text. , 1987 .

[12]  Yi-Chun Lin,et al.  Implementing clickers to assist learning in science lectures: The Clicker-Assisted Conceptual Change model , 2011 .

[13]  James Oigara,et al.  Interdisciplinary Journal of E-learning and Learning Objects Teaching and Learning with Clickers: Are Clickers Good for Students? , 2022 .

[14]  Scott Freeman,et al.  Prescribed active learning increases performance in introductory biology , 2007, CBE life sciences education.

[15]  J. Ana Donaldson,et al.  Engaging the Online Learner: Activities and Resources for Creative Instruction. Updated Edition. , 2004 .

[16]  Jane E Caldwell,et al.  Clickers in the large classroom: current research and best-practice tips. , 2007, CBE life sciences education.

[17]  Barbara J. Guzzetti,et al.  Promoting conceptual change in science: A comparative meta-analysis of instructional interventions from reading education and science education , 1993 .

[18]  M. Limón.  On the cognitive conflict as an instructional strategy for conceptual change: a critical appraisal , 2001 .

[19]  Jane Willis,et al.  Doing Task-Based Teaching , 2007 .

[20]  Larry Ambrose,et al.  The power of feedback. , 2002, Healthcare executive.

[21]  Robert A. Bartsch,et al.  Examining the Effects of an Electronic Classroom Response System on Student Engagement and Performance , 2011 .

[22]  Jeff Cain,et al.  An audience response system strategy to improve student motivation, attention, and feedback. , 2009, American journal of pharmaceutical education.

[23]  S. Thompson,et al.  Quantifying heterogeneity in a meta‐analysis , 2002, Statistics in medicine.

[24]  Erika Blood,et al.  Using Student Response Systems in Lecture-Based Instruction: Does It Change Student Engagement and Learning? , 2008 .

[25]  Chang Liu,et al.  The Influence of Clickers Use on Metacognition and Learning Outcomes in College English Classroom , 2014, Int. J. Inf. Commun. Technol. Educ..

[26]  Kerri-Lee Krause,et al.  Students’ engagement in first‐year university , 2008 .

[27]  Robert Rosenthal,et al.  WRITING META-ANALYTIC REVIEWS , 1995 .

[28]  Alaba Olaoluwakotansibe Agbatogun,et al.  Developing Learners' Second Language Communicative Competence through Active Learning: Clickers or Communicative Approach? , 2014, J. Educ. Technol. Soc..

[29]  Nilo Eric Marin.  The impact of a Classroom Performance System on learning gains in a biology course for science majors , 2013 .

[30]  Huey-Min Sun,et al.  Effects of User and System Characteristics on Perceived Usefulness and Perceived Ease of Use of the Web-Based Classroom Response System , 2012 .

[31]  G. Tataronis,et al.  Click it: assessment of classroom response systems in physician assistant education. , 2011, Journal of allied health.

[32]  Ian D. Beatty.  Transforming Student Learning with Classroom Communication Systems , 2005, physics/0508129.

[33]  Kevin C. Almeroth,et al.  Clickers in college classrooms: Fostering learning with questioning methods in large lecture classes , 2009 .

[34]  Henk G. Schmidt,et al.  Situational Interest and Academic Achievement in the Active-Learning Classroom. , 2011 .

[35]  Hugh A. Stoddard,et al.  A Controlled Study of Improvements in Student Exam Performance With the Use of an Audience Response System During Medical School Lectures , 2010, Academic medicine : journal of the Association of American Medical Colleges.

[36]  Julie M. Kiefer.  The effect of electronic response systems : relationship between perceptions and class performance, and difference by gender and academic ability , 2013 .

[37]  Clyde Freeman Herreid,et al.  Gender differences in student performance in large lecture classrooms using personal response systems (‘clickers’) with narrative case studies , 2012 .

[38]  Caroline Elliott,et al.  Using a personal response system in economics teaching , 2003 .

[39]  Lucinda Ann Lynch.  The Effects of Clickers on Math Achievement in 11th Grade Mathematics , 2013 .

[40]  John C. Nesbit,et al.  Animated and static concept maps enhance learning from spoken narration , 2013 .

[41]  Usenime M. Akpanudo,et al.  The Effect of Clickers in University Science Courses , 2012, Journal of Science Education and Technology.

[42]  E. Wit.  Who wants to be… The use of a personal response system in statistics teaching , 2003 .

[43]  Jane S. Vogler,et al.  Classroom Response Systems Facilitate Student Accountability, Readiness, and Learning , 2013 .

[44]  Dennis M. Shaffer,et al.  Evaluating the Effectiveness of a Personal Response System in the Classroom , 2009 .

[45]  Cynthia B Paschal,et al.  Formative assessment in physiology teaching using a wireless classroom communication system. , 2002, Advances in physiology education.

[46]  Bill Rogers,et al.  Context matters: increasing understanding with interactive Clicker Case studies , 2011 .

[47]  S. Morgan.  The Effect of Using Clickers in Higher Education Science Classrooms. , 2012 .

[48]  David L. Longworth,et al.  Making the continuing medical education lecture effective , 1998 .

[49]  William B Wood,et al.  Teaching more by lecturing less. , 2005, Cell biology education.

[50]  Jeffrey R. Stowell,et al.  Using Student Response Systems (“Clickers”) to Combat Conformity and Shyness , 2010 .

[51]  J. C. Gagne.  The impact of clickers in nursing education: a review of literature. , 2011 .

[52]  L. Hedges,et al.  Introduction to Meta‐Analysis , 2009, International Coaching Psychology Review.

[53]  Debra Filer,et al.  Everyone's Answering: Using Technology to Increase Classroom Participation , 2010, Nursing education perspectives.

[54]  K. Anthis,et al.  Is It the Clicker, or Is It the Question? Untangling the Effects of Student Response System Use , 2011 .

[55]  Helena Seli,et al.  "Clickers" and metacognition: A quasi-experimental comparative study about metacognitive self-regulation and use of electronic feedback devices , 2013, Comput. Educ..

[56]  R. Mayer,et al.  When learning is just a click away: Does simple user interaction foster deeper understanding of multimedia messages? , 2001 .

[57]  Jill A. Marshall,et al.  Classroom Response Systems: A Review of the Literature , 2006 .

[58]  James A. Kole,et al.  Conserving Time in the Classroom: The Clicker Technique , 2011, Quarterly journal of experimental psychology.

[59]  Meghan D. McAuliffe,et al.  Efficacy of Personal Response Systems (“Clickers”) in Large, Introductory Psychology Classes , 2008 .

[60]  Tracy A. Lavin,et al.  A Systematic Review and Meta-Analysis of the Cognitive Correlates of Bilingualism , 2010 .

[61]  C. Wieman,et al.  Why peer discussion improves student performance on in-class concept questions , 2009 .

[62]  Linda C. Ueltschy.  An Exploratory Study of Integrating Interactive Technology into the Marketing Curriculum , 2001 .

[63]  Alice F. Healy,et al.  The Clicker Technique: Cultivating Efficient Teaching and Successful Learning , 2013 .

[64]  B. Patterson,et al.  Evidence for teaching practice: the impact of clickers in a large classroom environment. , 2010, Nurse education today.

[65]  Charles R. Graham,et al.  Empowering or compelling reluctant participators using audience response systems , 2007 .

[66]  Olusola O. Adesope,et al.  Learning With Concept and Knowledge Maps: A Meta-Analysis , 2006 .

[67]  J. C. De Gagne,et al.  The impact of clickers in nursing education: a review of literature. , 2011, Nurse education today.

[68]  Jodie C. Steeley.  Clicking on the learning light for community college students , 2012 .

[69]  Marianne Fallon,et al.  High-Tech Versus Low-Tech Instructional Strategies , 2011 .

[70]  R. Hake.  Interactive-engagement vs Traditional Methods in Mechanics Instruction* , 1998 .

[71]  Carla Carnaghan,et al.  Investigating the Effects of Group Response Systems on Student Satisfaction, Learning and Engagement in Accounting Education , 2007 .

[72]  Michael R. McNally.  The effects of classroom response systems on student learning and engagement , 2012 .

[73]  Thomas Andre,et al.  Level of Adjunct Question, Type of Feedback, and Learning Concepts by Reading. , 1988 .

[74]  Shawn M. Keough.  Clickers in the Classroom , 2012 .

[75]  Ceren Tekkaya,et al.  Misconceptions as barrier to understanding biology , 2002 .

[76]  J. Stowell,et al.  Benefits of Electronic Audience Response Systems on Student Participation, Learning, and Emotion , 2007 .

[77]  Glen E. Robinson,et al.  Synthesis of Research on the Effects of Class Size. , 1990 .

[78]  J. Delafuente,et al.  The silver tsunami is coming: will pharmacy be swept away with the tide? , 2009, American journal of pharmaceutical education.

[79]  R. Orwin.  A fail-safe N for effect size in meta-analysis. , 1983 .

[80]  D. Doherty,et al.  The audience response system and knowledge gain: A prospective study , 2012, Medical teacher.

[81]  S. Schwartz,et al.  Leaving College: Rethinking the Causes and Cures of Student Attrition , 1987 .

[82]  Daiyo Sawada,et al.  Audience Response Systems: Insipid Contrivances or Inspiring Tools? , 2006 .

[83]  Douglas Duncan,et al.  Clickers in the Classroom: How to Enhance Science Teaching Using Classroom Response Systems , 2004 .

[84]  Ian D. Beatty,et al.  Designing effective questions for classroom response system teaching , 2005, physics/0508114.

[85]  Cande V Ananth,et al.  The influence of an audience response system on knowledge retention: an application to resident education. , 2005, American journal of obstetrics and gynecology.

[86]  E. Deci,et al.  Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions. , 2000, Contemporary educational psychology.

[87]  Louis Abrahamson.  A Brief History of Networked Classrooms: Effects, Cases, Pedagogy, and Implications , 2006 .

[88]  Matthew J Bassignani,et al.  Effect of an audience response system on resident learning and retention of lecture material. , 2008, AJR. American journal of roentgenology.

[89]  K. Fitzpatrick,et al.  Effect of personal response systems on student perception and academic performance in courses in a health sciences curriculum. , 2011, Advances in physiology education.

[90]  Selçuk Karaman,et al.  Effects of audience response systems on student achievement and long-term retention , 2011 .

[91]  A. C. Butler,et al.  The critical role of retrieval practice in long-term retention , 2011, Trends in Cognitive Sciences.

[92]  P. Winne,et al.  Feedback and Self-Regulated Learning: A Theoretical Synthesis , 1995 .

[93]  Gerald Albaum,et al.  Classroom Questioning with Immediate Electronic Response: Do Clickers Improve Learning?. , 2008 .

[94]  J. Roschelle,et al.  Misconceptions Reconceived: A Constructivist Analysis of Knowledge in Transition , 1994 .

[95]  Robert M. Gonyea,et al.  Learning at a Distance: Engaged or Not? , 2008 .

[96]  Cheryl Fortner-Wood,et al.  The Effects of Student Response Systems on Student Learning and Attitudes in Undergraduate Psychology Courses , 2013 .

[97]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement , 2009, BMJ.

[98]  Robin H. Kay,et al.  Examining the benefits and challenges of using audience response systems: A review of the literature , 2009, Comput. Educ..

[99]  Matti Uhari,et al.  Experiences of using an interactive audience response system in lectures , 2003, BMC medical education.

[100]  K. Hyun Lim.  Addressing the multiplication makes bigger and division makes smaller misconceptions via prediction and clickers , 2011 .

[101]  L. Hedges,et al.  Statistical Methods for Meta-Analysis , 1987 .

[102]  Barbara Petersohn,et al.  Classroom Performance Systems, Library Instruction, and Instructional Design: A Pilot Study , 2008 .

[103]  Janice Berry,et al.  TECHNOLOGY SUPPORT in Nursing Education: Clickers in the Classroom , 2009, Nursing education perspectives.

[104]  J. Sterne,et al.  Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis. , 2001, Journal of clinical epidemiology.

[105]  Alice Richardson,et al.  Instructor perceptions of using a mobile-phone-based free classroom response system in first-year statistics undergraduate courses , 2012 .

[106]  S Duval,et al.  Trim and Fill: A Simple Funnel‐Plot–Based Method of Testing and Adjusting for Publication Bias in Meta‐Analysis , 2000, Biometrics.

[107]  Kirsten Crossgrove,et al.  Using clickers in nonmajors- and majors-level biology courses: student opinion, learning, and long-term retention of course material. , 2008, CBE life sciences education.

[108]  G. Glass,et al.  Meta-analysis in social research , 1981 .

[109]  Jennifer A. Fredricks,et al.  School Engagement: Potential of the Concept, State of the Evidence , 2004 .

[110]  Mary Jo Wagner,et al.  Not another boring lecture: engaging learners with active learning techniques. , 2015, The Journal of emergency medicine.

[111]  Emily K Chan,et al.  “Clicking” with your audience: Evaluating the use of personal response systems in library instruction , 2010 .

[112]  T Eric Schackow,et al.  Audience response system: effect on learning in family medicine residents. , 2004, Family medicine.

[113]  George D. Kuh,et al.  Adding Value: Learning Communities and Student Engagement , 2004 .

[114]  Roberta Sullivan,et al.  Principles for Constructing Good Clicker Questions: Going beyond Rote Learning and Stimulating Active Engagement with Course Content , 2009 .

[115]  Lorena Blasco-Arcas,et al.  Using clickers in class. The role of interactivity, active collaborative learning and engagement in learning performance , 2013, Comput. Educ..

[116]  Gerald Bergtrom,et al.  Clicker Sets as Learning Objects , 2006 .