Conditions for the Effectiveness of Multiple Visual Representations in Enhancing STEM Learning

Visual representations play a critical role in enhancing science, technology, engineering, and mathematics (STEM) learning. Educational psychology research shows that adding visual representations to text can enhance students’ learning of content knowledge, compared to text-only. But should students learn with a single type of visual representation or with multiple different types of visual representations? This article addresses this question from the perspective of the representation dilemma, namely that students often learn content they do not yet understand from representations they do not yet understand. To benefit from visual representations, students therefore need representational competencies, that is, knowledge about how visual representations depict information about the content. This article reviews literature on representational competencies involved in students’ learning of content knowledge. Building on this review, this article analyzes how the number of visual representations affects the role these representational competencies play during students’ learning of content knowledge. To this end, the article compares two common scenarios: text plus a single type of visual representations (T+SV) and text plus multiple types of visual representations (T+MV). The comparison yields seven hypotheses that describe under which conditions T+MV scenarios are more effective than T+SV scenarios. Finally, the article reviews empirical evidence for each hypothesis and discusses open questions about the representation dilemma.

[1]  Paul J. Feltovich,et al.  Categorization and Representation of Physics Problems by Experts and Novices , 1981, Cogn. Sci..

[2]  Lorelei Lingard,et al.  ‘You see?’ Teaching and learning how to interpret visual cues during surgery , 2015, Medical Education.

[3]  Tina Seufert Supporting Coherence Formation in Learning from Multiple Representations , 2003 .

[4]  Albert T. Corbett,et al.  The Knowledge-Learning-Instruction Framework: Bridging the Science-Practice Chasm to Enhance Robust Student Learning , 2012, Cogn. Sci..

[5]  James G. Greeno,et al.  Learning in Activity , 2014 .

[6]  G. Olympiou,et al.  Blending Physical and Virtual Manipulatives: An Effort to Improve Students' Conceptual Understanding through Science Laboratory Experimentation , 2012 .

[7]  Andreas Holzinger,et al.  Dynamic Media in Computer Science Education; Content Complexity and Learning Performance: Is Less More? , 2008, J. Educ. Technol. Soc..

[8]  Hans Spada,et al.  Acquiring knowledge in science and mathematics : the use of multiple representations in technology based learning environments , 1998 .

[9]  Canan Nakiboğlu,et al.  INSTRUCTIONAL MISCONCEPTIONS OF TURKISH PROSPECTIVE CHEMISTRY TEACHERS ABOUT ATOMIC ORBITALS AND HYBRIDIZATION , 2003 .

[10]  John Airey,et al.  A disciplinary discourse perspective on university science learning: Achieving fluency in a critical constellation of modes , 2009 .

[11]  Rayne A. Sperling,et al.  The effects of levels of elaboration on learners’ strategic processing of text , 2011 .

[12]  Charalambos Y. Charalambous,et al.  Drawing on a Theoretical Model to Study Students’ Understandings of Fractions , 2007 .

[13]  Katharina Scheiter,et al.  How Inspecting a Picture Affects Processing of Text in Multimedia Learning , 2013 .

[14]  Konrad J. Schönborn,et al.  Experts' Views on Translation Across Multiple External Representations in Acquiring Biological Knowledge About Ecology, Genetics, and Evolution , 2013 .

[15]  A. Bandura Social cognitive theory: an agentic perspective. , 1999, Annual review of psychology.

[16]  Lieven Verschaffel,et al.  Students’ reported justifications for their representational choices in linear function problems: an interview study , 2013 .

[17]  D. Uttal,et al.  The malleability of spatial skills: a meta-analysis of training studies. , 2013, Psychological bulletin.

[18]  J. Roschelle Learning by Collaborating: Convergent Conceptual Change , 1992 .

[19]  Martina A. Rau,et al.  Connection making between multiple graphical representations: A multi-methods approach for domain-specific grounding of an intelligent tutoring system for chemistry , 2015, Comput. Educ..

[20]  S. Ainsworth DeFT: A Conceptual Framework for Considering Learning with Multiple Representations. , 2006 .

[21]  Richard E. Mayer,et al.  The Cambridge Handbook of Multimedia Learning: Cognitive Theory of Multimedia Learning , 2005 .

[22]  Y. Lincoln,et al.  Scientific Research in Education , 2004 .

[23]  Stacey Lowery Bretz,et al.  Generating cognitive dissonance in student interviews through multiple representations , 2012 .

[24]  N. Finkelstein,et al.  Patterns of multiple representation use by experts and novices during physics problem solving , 2008 .

[25]  P. Thompson,et al.  Fractions and multiplicative reasoning , 2003 .

[26]  Ruth Wylie,et al.  The Self-Explanation Principle in Multimedia Learning , 2014 .

[27]  Roger Azevedo,et al.  The Cambridge Handbook of the Learning Sciences: Metacognition , 2014 .

[28]  D. J. Gilmore,et al.  Individual differences and strategy selection in reasoning , 1997 .

[29]  Ruhama Even,et al.  Subject matter knowledge for teaching and the case of functions , 1990 .

[30]  Robert B. Kozma,et al.  Students Becoming Chemists: Developing Representationl Competence , 2005 .

[31]  K. Anders Ericsson Perceptual and Memory Processes in the Acquisition of Expert Performance: The EPAM Model , 2014 .

[32]  David F. Treagust,et al.  Towards a Coherent Model for Macro, Submicro and Symbolic Representations in Chemical Education , 2009 .

[33]  M. Just,et al.  Constructing mental models of machines from text and diagrams. , 1993 .

[34]  Roy D. Pea,et al.  Distributed by Design: On the Promises and Pitfalls of Collaborative Learning with Multiple Representations , 2011 .

[35]  Nicole M. McNeil,et al.  Should you show me the money? Concrete objects both hurt and help performance on mathematics problems , 2009 .

[36]  David W. Carraher,et al.  The Transfer Dilemma , 2002 .

[37]  Vincent Aleven,et al.  Interleaved Practice in Multi-Dimensional Learning Tasks: Which Dimension Should We Interleave?. , 2013 .

[38]  Ton de Jong,et al.  The effects of directive self-explanation prompts to support active processing of multiple representations in a simulation-based learning environment , 2011, J. Comput. Assist. Learn..

[39]  Assaf Harel,et al.  What is special about expertise? Visual expertise reveals the interactive nature of real-world object recognition , 2016, Neuropsychologia.

[40]  A. Shusterman,et al.  Teaching Chemistry with Electron Density Models , 1997 .

[41]  S. Paris,et al.  Children's Metacognition About Reading: issues in Definition, Measurement, and Instruction , 1987 .

[42]  A. Northedge Enabling Participation in Academic Discourse , 2003 .

[43]  Beate Grawemeyer,et al.  Evaluation of ERST - An External Representation Selection Tutor , 2006, Diagrams.

[44]  Marios Papaevripidou,et al.  Effects of experimenting with physical and virtual manipulatives on students' conceptual understanding in heat and temperature , 2008 .

[45]  C. Hartshorne,et al.  Collected Papers of Charles Sanders Peirce , 1935, Nature.

[46]  David H. Uttal,et al.  Comprehending and Learning from ‘Visualizations’: A Developmental Perspective , 2008 .

[47]  Barbara Y. White,et al.  Making Their Own Connections: Students' Understanding of Multiple Models in Basic Electricity , 1999 .

[48]  Michelene T. H. Chi,et al.  Self-Explanations: How Students Study and Use Examples in Learning To Solve Problems. Technical Report No. 9. , 1987 .

[49]  Peter Gerjets,et al.  Extending multimedia research: How do prerequisite knowledge and reading comprehension affect learning from text and pictures , 2014, Comput. Hum. Behav..

[50]  Teaching the Concept of Unit in Measurement Interpretation of Rational Numbers , 2008 .

[51]  Yuri Uesaka,et al.  Active Comparison as a Means of Promoting the Development of Abstract Conditional Knowledge and Appropriate Choice of Diagrams in Math Word Problem Solving , 2006, Diagrams.

[52]  M. Chi,et al.  Eliciting Self‐Explanations Improves Understanding , 1994 .

[53]  A. Gagatsis,et al.  Representational Flexibility and Problem-Solving Ability in Fraction and Decimal Number Addition: A Structural Model , 2016 .

[54]  E. Wiebe,et al.  The influence of prior knowledge on viewing and interpreting graphics with macroscopic and molecular representations , 2008 .

[55]  Barbara Tversky,et al.  Visualizing Thought , 2011, Top. Cogn. Sci..

[56]  Trevor R Anderson,et al.  The importance of visual literacy in the education of biochemists * , 2006, Biochemistry and molecular biology education : a bimonthly publication of the International Union of Biochemistry and Molecular Biology.

[57]  John K. Gilbert,et al.  Visualization: A Metacognitive Skill in Science and Science Education , 2005 .

[58]  P. Pintrich A Motivational Science Perspective on the Role of Student Motivation in Learning and Teaching Contexts. , 2003 .

[59]  R. Cox Representation construction, externalised cognition and individual differences , 1999 .

[60]  Jan Elen,et al.  Understanding and Enhancing the Use of Multiple External Representations in Chemistry Education , 2012 .

[61]  Michael L. Eastwood,et al.  Fastest Fingers: A Molecule-Building Game for Teaching Organic Chemistry , 2013 .

[62]  Jean McKendree,et al.  The Role of Representation in Teaching and Learning Critical Thinking , 2002 .

[63]  Miriam Reiner,et al.  Visualization : theory and practice in science education , 2008 .

[64]  Russell N. Carney,et al.  Pictorial Illustrations Still Improve Students' Learning from Text , 2002 .

[65]  Kathleen A. Cramer,et al.  Using manipulative models to build number sense for addition of fractions: NCTM 2002 yearbook , 2002 .

[66]  Rosária Justi,et al.  Models and Modelling in Chemical Education , 2002 .

[67]  Gayle Nicoll,et al.  A report of undergraduates' bonding misconceptions , 2001 .

[68]  Jay R. Campbell,et al.  The Nation's Report Card: Reading, 2002. , 2003 .

[69]  Kathleen A. Cramer,et al.  Efficacy of Different Concrete Models for Teaching the Part-Whole Construct for Fractions , 2009 .

[70]  Stephen J. Pape,et al.  The Role of Representation(s) in Developing Mathematical Understanding , 2001 .

[71]  David F. Feldon,et al.  Five common but questionable principles ofmultimedia learning , 2005 .

[72]  John Hattie,et al.  Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement , 2008 .

[73]  Helen Crompton,et al.  Investigating how college students’ task definitions and plans relate to self-regulated learning processing and understanding of a complex science topic , 2012 .

[74]  Slava Kalyuga,et al.  Rethinking the Boundaries of Cognitive Load Theory in Complex Learning , 2016 .

[75]  Brian J. Reiser,et al.  The Cambridge Handbook of the Learning Sciences: Scaffolding , 2014 .

[76]  R. Säljö,et al.  Expertise Differences in the Comprehension of Visualizations: a Meta-Analysis of Eye-Tracking Research in Professional Domains , 2011 .

[77]  Matthew Heinsen Egan,et al.  Program visualization and explanation for novice C programmers , 2014, ACE.

[78]  Melanie M. Cooper,et al.  Development and validation of the implicit information from Lewis structures instrument (IILSI): do students connect structures with properties? , 2012 .

[79]  L. Verschaffel,et al.  Improving students’ representational flexibility in linear-function problems: an intervention , 2014 .

[80]  Hope J. Hartman Metacognition in Learning and Instruction , 2001 .

[81]  Rayane F. Moreira A Game for the Early and Rapid Assimilation of Organic Nomenclature , 2013 .

[82]  R. Kozma,et al.  Multimedia and understanding: Expert and novice responses to different representations of chemical phenomena , 1997 .

[83]  J. Shea National Science Education Standards , 1995 .

[84]  Sibel Kazak,et al.  Saying More than You Know in Instructional Settings , 2011 .

[85]  M. Stieff Mental rotation and diagrammatic reasoning in science , 2007 .

[86]  E. Gibson Principles of Perceptual Learning and Development , 1969 .

[87]  Keith S. Taber,et al.  An alternative conceptual framework from chemistry education , 1998 .

[88]  Albert T. Corbett,et al.  The Resilience of Overgeneralization of Knowledge about Data Representations. , 2002 .

[89]  Herbert A. Simon,et al.  Why a Diagram is (Sometimes) Worth Ten Thousand Words , 1987 .

[90]  Michal Yerushalmy,et al.  Student perceptions of aspects of algebraic function using multiple representation software , 1991 .

[91]  A. Renkl,et al.  Instructional Aids to Support a Conceptual Understanding of Multiple Representations. , 2009 .

[92]  Edward C. Rathmell,et al.  Implementing the Standards. Number Representations and Relationships. , 1991 .

[93]  H. Simon,et al.  Perception in chess , 1973 .

[94]  Irena Sajovic,et al.  Action Research to Promote the Formation of Linkages by Chemistry Students Between the Macro, Submicro, and Symbolic Representational Levels , 2009 .

[95]  Shaaron Ainsworth,et al.  Examining the Effects of Different Multiple Representational Systems in Learning Primary Mathematics , 2002 .

[96]  Wolfgang Schnotz,et al.  Interactive and Non-Interactive Pictures in Multimedia Learning Environments: Effects on Learning Outcomes and Learning Efficiency , 2009 .

[97]  Keith S. Taber,et al.  Revisiting the chemistry triplet: drawing upon the nature of chemical knowledge and the psychology of learning to inform chemistry education , 2013 .

[98]  Robert L. Goldstone,et al.  The Transfer of Scientific Principles Using Concrete and Idealized Simulations , 2005, Journal of the Learning Sciences.

[99]  Vincent Aleven,et al.  Complementary Effects of Sense-Making and Fluency-Building Support for Connection Making: A Matter of Sequence? , 2013, AIED.

[100]  Zachary A. Pardos,et al.  How Should Intelligent Tutoring Systems Sequence Multiple Graphical Representations of Fractions? A Multi-Methods Study , 2013, International Journal of Artificial Intelligence in Education.

[101]  J. Deloache Dual representation and young children's use of scale models. , 2000, Child development.

[102]  P. Shah,et al.  Exploring visuospatial thinking in chemistry learning , 2004 .

[103]  Philip J. Kellman,et al.  Perceptual Learning and the Technology of Expertise: Studies in Fraction Learning and Algebra. , 2008 .

[104]  Maria Caterina Tornatora,et al.  An Eye-Tracking Study of Learning From Science Text With Concrete and Abstract Illustrations , 2013 .

[105]  E. Gibson Perceptual Learning in Development: Some Basic Concepts , 2000 .

[106]  Etienne Wenger,et al.  Situated Learning: Legitimate Peripheral Participation , 1991 .

[107]  Ghislain Deslongchamps,et al.  When do spatial abilities support student comprehension of STEM visualizations? , 2013, Cognitive Processing.

[108]  Romina M. J. Proctor,et al.  Integrating concrete and virtual materials in an elementary mathematics classroom: a case study of success with fractions , 2002 .

[109]  Z. Zacharia,et al.  Physical and Virtual Laboratories in Science and Engineering Education , 2013, Science.

[110]  George M. Bodner,et al.  Mental Models : The Role of Representations in Problem Solving in Chemistry PROCEEDINGS , 2002 .

[111]  P. Donahue,et al.  The Nation's Report Card[TM]: Reading, 2003. NCES 2005-453. , 2005 .

[112]  Johnna J. Bolyard,et al.  What Are Virtual Manipulatives , 2002 .

[113]  Ruhama Even,et al.  Factors involved in linking representations of functions , 1998 .

[114]  L. Verschaffel,et al.  What counts as a flexible representational choice? An evaluation of students’ representational choices to solve linear function problems , 2012 .

[115]  P. Chandler,et al.  Cognitive Load Theory and the Format of Instruction , 1991 .

[116]  R. Mayer,et al.  Nine Ways to Reduce Cognitive Load in Multimedia Learning , 2003 .

[117]  John K. Gilbert,et al.  Visualization: An Emergent Field of Practice and Enquiry in Science Education , 2008 .

[118]  Martin Reisslein,et al.  Teaching with Concrete and Abstract Visual Representations: Effects on Students' Problem Solving, Problem Representations, and Learning Perceptions. , 2011 .

[119]  Judy S. DeLoache,et al.  When a picture is not worth a thousand words: Young children's understanding of pictures and models , 1992 .

[120]  Emily R. Fyfe,et al.  “Concreteness fading” promotes transfer of mathematical knowledge , 2012 .

[121]  Linda Jarvin,et al.  When Theories Don't Add Up: Disentangling he Manipulatives Debate , 2007 .

[122]  L. Verschaffel,et al.  Representational flexibility in linear-function problems: a choice/no-choice study , 2010 .

[123]  A. Northedge Organizing Excursions Into Specialist Discourse Communities: A Sociocultural Account of University Teaching , 2008 .

[124]  Thomas E. Kieren,et al.  Rational and fractional numbers: From quotient fields to recursive understanding , 1993 .

[125]  Billie Eilam,et al.  Possible Constraints of Visualization in Biology: Challenges in Learning with Multiple Representations , 2013 .

[126]  R. Kozma,et al.  The Roles of Representations and Tools in the Chemistry Laboratory and Their Implications for Chemistry Learning , 2000 .

[127]  P. Pintrich Multiple Goals, Multiple Pathways: The Role of Goal Orientation in Learning and Achievement. , 2000 .

[128]  James V. Wertsch Properties of Mediated Action , 1997 .

[129]  Mireille Betrancourt,et al.  The Cambridge Handbook of Multimedia Learning: The Animation and Interactivity Principles in Multimedia Learning , 2005 .

[130]  C. Hoyles,et al.  The Construction of Mathematical Meanings: Connecting the Visual with the Symbolic , 1997 .

[131]  Vincent Aleven,et al.  Successful learning with multiple graphical representations and self-explanation prompts. , 2015 .

[132]  Robert L. Goldstone,et al.  Concreteness Fading in Mathematics and Science Instruction: a Systematic Review , 2014 .

[133]  T. Mueller Learning to Think Spatially , 2006 .

[134]  V. Aleven,et al.  Triggering situational interest by decorative illustrations both fosters and hinders learning in computer-based learning environments , 2014 .

[135]  Vincent Aleven,et al.  Sense Making Alone Doesn't Do It: Fluency Matters Too! ITS Support for Robust Learning with Multiple Representations , 2012, ITS.

[136]  Rosária Justi,et al.  The Application of a ‘Model of Modelling’ to Illustrate the Importance of Metavisualisation in Respect of the Three Types of Representation , 2009 .

[137]  Paul Cobb,et al.  Guiding Inquiry-Based Math Learning , 2005 .

[138]  M. Bannert,et al.  Construction and interference in learning from multiple representation , 2003 .

[139]  R. Mayer,et al.  Eight Ways to Promote Generative Learning , 2016 .

[140]  Joseph K. Torgesen,et al.  Fraction Skills and Proportional Reasoning , 2007 .

[141]  A. Rocke Image and Reality: Kekulé, Kopp, and the Scientific Imagination , 2010 .

[142]  A. Gagatsis,et al.  The Effects of Different Modes of Representation on Mathematical Problem Solving. , 2004 .

[143]  Phillip L. Ackerman,et al.  Cognitive Ability and Non-Ability Trait Determinants of Expertise , 2003 .

[144]  Susan E. Newman,et al.  Cognitive Apprenticeship: Teaching the Craft of Reading, Writing, and Mathematics. Technical Report No. 403. , 1987 .

[145]  Ghislain Deslongchamps,et al.  Beyond ball-and-stick: Students' processing of novel STEM visualizations , 2013 .

[146]  Martha W. Alibali,et al.  Modal Engagements in Pre-College Engineering: Tracking Math and Science Concepts Across Symbols, Sketches, Software, Silicone, and Wood , 2011 .

[147]  John A. Hortin,et al.  Identifying the Theoretical Foundations of Visual Literacy. , 1982 .

[148]  Vicente A Talanquer,et al.  Commonsense Chemistry: A Model for Understanding Students' Alternative Conceptions , 2006 .

[149]  Ji Y. Son,et al.  Perceptual Learning Modules in Mathematics: Enhancing Students' Pattern Recognition, Structure Extraction, and Fluency , 2010, Top. Cogn. Sci..

[150]  Billie Eilam,et al.  Teaching, Learning, and Visual Literacy: The Dual Role of Visual Representation , 2012 .

[151]  Jere Confrey,et al.  The construction, refinement, and early validation of the equipartitioning learning trajectory , 2010, ICLS.

[152]  Andrea A. diSessa,et al.  Meta-representation: an introduction , 2000 .

[153]  C. Furió,et al.  Functional fixedness and functional reduction as common sense reasonings in chemical equilibrium and in geometry and polarity of molecules , 2000 .

[154]  P. Kellman,et al.  Perceptual Learning, Cognition, and Expertise , 2013 .

[155]  D. Gentner Structure‐Mapping: A Theoretical Framework for Analogy* , 1983 .

[156]  D. Gentner,et al.  Learning and Transfer: A General Role for Analogical Encoding , 2003 .

[157]  A. Baddeley Working memory: theories, models, and controversies. , 2012, Annual review of psychology.

[158]  P. V. Meter,et al.  The Promise and Practice of Learner-Generated Drawing: Literature Review and Synthesis , 2005 .

[159]  Jennifer L. Chiu,et al.  Evidence for effective uses of dynamic visualisations in science curriculum materials , 2015 .

[160]  Robert L. Goldstone,et al.  Learning to Bridge Between Perception and Cognition , 1997 .

[161]  Daniel Bodemer,et al.  External and mental referencing of multiple representations , 2006, Comput. Hum. Behav..

[162]  Robert L. Goldstone,et al.  Integrating Formal and Grounded Representations in Combinatorics Learning , 2013 .

[163]  Bette Davidowitz,et al.  Linking the macroscopic and sub-microscopic levels : diagrams , 2009 .

[164]  Paul Brna,et al.  Supporting the use of external representation in problem solving: the need for flexible learning environments , 1995 .

[165]  Wolfgang Schnotz,et al.  The Cambridge Handbook of Multimedia Learning: An Integrated Model of Text and Picture Comprehension , 2005 .

[166]  S. Ainsworth The Cambridge Handbook of Multimedia Learning: The Multiple Representation Principle in Multimedia Learning , 2014 .

[167]  Adam Kraft,et al.  What happens when representations fail to represent? Graduate students’ mental models of organic chemistry diagrams , 2010 .

[168]  D. Gentner,et al.  Structure mapping in analogy and similarity. , 1997 .

[169]  Robert L. Goldstone,et al.  Reuniting perception and conception , 1998, Cognition.

[170]  A. Paivio Mental Representations: A Dual Coding Approach , 1986 .

[171]  P. Kellman,et al.  Perceptual learning and human expertise. , 2009, Physics of life reviews.

[172]  R. Keith Sawyer,et al.  The Cambridge Handbook of the Learning Sciences: Analyzing Collaborative Discourse , 2005 .

[173]  Lester C. Loschky,et al.  How Does Visual Attention Differ Between Experts and Novices on Physics Problems , 2010 .

[174]  Dor Abrahamson,et al.  Understanding ratio and proportion as an example of the apprehending zone and conceptual-phase problem-solving models. , 2004 .

[175]  M. Hegarty,et al.  Individual Differences in Spatial Abilities , 2005 .

[176]  Paul Cobb,et al.  Cultural Tools and Mathematical Learning: A Case Study. , 1995 .

[177]  David Pace,et al.  Decoding the Disciplines: A Model for Helping Students Learn Disciplinary Ways of Thinking. , 2004 .

[178]  Yash Patel,et al.  Using Multiple Representations to Build Conceptual Understanding in Science and Mathematics , 2014 .

[179]  Manjula D. Sharma,et al.  Scientific representational fluency: Defining, diagnosing and developing , 2014 .

[180]  Anna N. Rafferty,et al.  Designing Instruction to Improve Lifelong Inquiry Learning , 2015 .

[181]  Joan K. Gallini,et al.  When Is an Illustration Worth Ten Thousand Words , 1990 .

[182]  Patrik Pluchino,et al.  Effects of Picture Labeling on Science Text Processing and Learning: Evidence From Eye Movements , 2013 .

[183]  David F. Treagust,et al.  Investigation of secondary school, undergraduate, and graduate learners' mental models of ionic bonding , 2003 .

[184]  Abraham Arcavi,et al.  The role of visual representations in the learning of mathematics , 2003 .

[185]  Shaaron Ainsworth,et al.  The Educational Value of Multiple-representations when Learning Complex Scientific Concepts , 2008 .

[186]  C. W. Bowen,et al.  Representational Systems Used by Graduate Students while Problem Solving in Organic Synthesis. , 1990 .