Representational competence: towards a distributed and embodied cognition account

Abstract Multiple external representations (MERs) are central to the practice and learning of science, mathematics and engineering, as the phenomena and entities investigated and controlled in these domains are often not available for perception and action. MERs therefore play a twofold constitutive role in reasoning in these domains. Firstly, MERs stand in for the phenomena and entities that are imagined, and thus make possible scientific investigations. Secondly, related to the above, sensorimotor and imagination-based interactions with the MERs make possible focused cognitive operations involving these phenomena and entities, such as mental rotation and analogical transformations. These two constitutive roles suggest that acquiring expertise in science, mathematics and engineering requires developing the ability to transform and integrate the MERs in that field, in tandem with running operations in imagination on the phenomena and entities the MERs stand for. This core ability to integrate external and internal representations and operations on them – termed representational competence (RC) – is therefore critical to learning in science, mathematics and engineering. However, no general account of this core process is currently available. We argue that, given the above two constitutive roles played by MERs, a theoretical account of representational competence requires an explicit model of how the cognitive system interacts with external representations, and how imagination abilities develop through this process. At the applied level, this account is required to develop design guidelines for new media interventions for learning science and mathematics, particularly emerging ones that are based on embodied interactions. As a first step to developing such a theoretical account, we review the literature on learning with MERs, as well as acquiring RC, in chemistry, biology, physics, mathematics and engineering, from two perspectives. First, we focus on the important theoretical accounts and related empirical studies, and examine what is common about them. Second, we summarise the major trends in each discipline, and then bring together these trends. The results show that most models and empirical studies of RC are framed within the classical information processing approach, and do not take a constitutive view of external representations. To develop an account compatible with the constitutive view of external representations, we outline an interaction-based theoretical account of RC, extending recent advances in distributed and embodied cognition.

[1]  C. Kamii,et al.  The difficulty of “length × width”: Is a square the unit of measurement? , 2006 .

[2]  A. Paivio Dual coding theory: Retrospect and current status. , 1991 .

[3]  Hans Spada,et al.  The Active Integration of Information during Learning with Dynamic and Interactive Visualisations , 2004 .

[4]  Ton de Jong,et al.  Learning with Multiple Representations. Advances in Learning and Instruction Series. , 1998 .

[5]  Jules M. Zimmer,et al.  Cross-National Comparison of Representational Competence. , 1999 .

[6]  Sanjay Chandrasekharan,et al.  Beyond Telling: Where New Computational Media is Taking Model-Based Reasoning , 2016 .

[7]  Nancy J. Nersessian,et al.  Turning Experiments into Objects: The Cognitive Processes Involved in the Design of a Lab‐on‐a‐Chip Device , 2013 .

[8]  RogersYvonne,et al.  Introduction to the special issue on the theory and practice of embodied interaction in HCI and interaction design , 2013 .

[9]  Ton de Jong,et al.  Scientific Discovery Learning with Computer Simulations of Conceptual Domains , 1998 .

[10]  Stephen J. Pape,et al.  The Role of Representation(s) in Developing Mathematical Understanding , 2001 .

[11]  Mary B. Nakhleh,et al.  Learning Chemistry Using Multiple External Representations , 2008 .

[12]  Shaaron Ainsworth,et al.  The Educational Value of Multiple-representations when Learning Complex Scientific Concepts , 2008 .

[13]  Seventh Grade Students' Use of Multiple Representations in Pattern Related Algebra Tasks. , 2006 .

[14]  Uri Wilensky,et al.  GasLab—an Extensible Modeling Toolkit for Connecting Micro-and Macro-properties of Gases , 1999 .

[15]  Logan Fiorella,et al.  Principles for Reducing Extraneous Processing in Multimedia Learning: Coherence, Signaling, Redundancy, Spatial Contiguity and Temporal Contiguity Principles. , 2014 .

[16]  P Viviani,et al.  Perceptual anticipation in handwriting: The role of implicit motor competence , 2000, Perception & psychophysics.

[17]  R. Pea Practices of distributed intelligence and designs for education , 2018 .

[18]  Aditi KOTHIYALa,et al.  How does representational competence develop ? Explorations using a fully controllable interface and eye-tracking , 2015 .

[19]  M. Hegarty,et al.  Models as Feedback: Developing Representational Competence in Chemistry. , 2015 .

[20]  B. Sherin,et al.  Inventing Graphing: Meta- Representational Expertise in Children , 2007 .

[21]  Andra A. DiSessa Inventing Graphing: Meta­ Representational Expertise in Children , 1991 .

[22]  Katherine K. Perkins,et al.  Making On-line Science Course Materials Easily Translatable and Accessible Worldwide: Challenges and Solutions , 2012 .

[23]  C. Tsui,et al.  Genetics Reasoning with Multiple External Representations , 2003 .

[24]  James J. Kaput,et al.  A Research Base Supporting Long Term Algebra Reform , 1995 .

[25]  Gaea Leinhardt,et al.  Functions, Graphs, and Graphing: Tasks, Learning, and Teaching , 1990 .

[26]  Pablo Neruda,et al.  Rethinking Situatedness : Environment Structure in the Time of the Common Code , 2010 .

[27]  The Analysis of Teacher Candidates\' Translating Skills in Multiple Representations , 2012 .

[28]  Michael W. Briggs,et al.  A Model of Molecular Visualization , 2005 .

[29]  Patrick Dillon,et al.  The Effects of Scaffolded Simulation-Based Inquiry Learning on Fifth-Graders' Representations of the Greenhouse Effect , 2014 .

[30]  Richard E. Mayer,et al.  The Cambridge Handbook of Multimedia Learning: Principles for Reducing Extraneous Processing in Multimedia Learning : Coherence, Signaling, Redundancy, Spatial Contiguity, and Temporal Contiguity Principles , 2005 .

[31]  W.M. McCracken,et al.  Text to diagram to symbol: representational transformations in problem-solving , 2001, 31st Annual Frontiers in Education Conference. Impact on Engineering and Science Education. Conference Proceedings (Cat. No.01CH37193).

[32]  P. Shah,et al.  Exploring visuospatial thinking in chemistry learning , 2004 .

[33]  Alex H. Johnstone,et al.  TEACHING OF CHEMISTRY - LOGICAL OR PSYCHOLOGICAL? , 2000 .

[34]  David Kirsh,et al.  Embodied cognition and the magical future of interaction design , 2013, TCHI.

[35]  A. diSessa Metarepresentation: Native Competence and Targets for Instruction , 2004 .

[36]  Cindy E. Hmelo-Silver,et al.  Comparing expert and novice understanding of a complex system from the perspective of structures, behaviors, and functions , 2004, Cogn. Sci..

[37]  D. Bub,et al.  On the dynamics of action representations evoked by names of manipulable objects. , 2012, Journal of experimental psychology. General.

[38]  Ann C.H. Kindfield,et al.  Biology Diagrams: Tools to Think With , 1994 .

[39]  P.K. Imbrie,et al.  A framework for posing open-ended engineering problems: model-eliciting activities , 2004, 34th Annual Frontiers in Education, 2004. FIE 2004..

[40]  Merlyn J. Behr,et al.  Representations and translations among representations in mathematics learning and problem solving , 1987 .

[41]  R. Schubotz Prediction of external events with our motor system: towards a new framework , 2007, Trends in Cognitive Sciences.

[42]  Y. Anzai Learning and use of representations for physics expertise , 1991 .

[43]  Shamin Padalkar,et al.  Visualisation and Reasoning in Explaining the Phases of the Moon , 2009 .

[44]  William B. Jensen,et al.  Logic, History, and the Chemistry Textbook I. Does Chemistry Have a Logical Structure? , 1998 .

[45]  R. Cox Representation construction, externalised cognition and individual differences , 1999 .

[46]  Sanjay Chandrasekharan,et al.  The cognitive science of Feynmen , 2013 .

[47]  Loretta L. Jones,et al.  The role of multiple representations in the understanding of ideal gas problems , 2011 .

[48]  Pierre Dillenbourg,et al.  Collaborative Learning: Cognitive and Computational Approaches. Advances in Learning and Instruction Series. , 1999 .

[49]  E. Knuth Student Understanding of the Cartesian Connection: An Exploratory Study , 2000 .

[50]  Sascha Bernholt,et al.  Use of representation mapping to capture abstraction in problem solving in different courses in chemistry , 2015 .

[51]  Carolyn Kieran Concepts associated with the equality symbol , 1981 .

[52]  Joseph Krajcik,et al.  Promoting understanding of chemical representations: Students' use of a visualization tool in the classroom , 2001 .

[53]  David Landy,et al.  A perceptual account of symbolic reasoning , 2014, Front. Psychol..

[54]  Sanjay Chandrasekharan Rational Intuition: Becoming Knowledge: Cognitive and Neural Mechanisms That Support Scientific Intuition , 2014 .

[55]  A. Johnstone Why is science difficult to learn? Things are seldom what they seem , 1991 .

[56]  George M. Bodner,et al.  Mental Models : The Role of Representations in Problem Solving in Chemistry PROCEEDINGS , 2002 .

[57]  M. Cole A cultural-historical approach to distributed cognition , 1993 .

[58]  Russell Tytler,et al.  A Framework for Re‐thinking Learning in Science from Recent Cognitive Science Perspectives , 2010 .

[59]  P. Chandler,et al.  Evidence for Cognitive Load Theory , 1991 .

[60]  Martin Reisslein,et al.  Pre‐college Electrical Engineering Instruction: The Impact of Abstract vs. Contextualized Representation and Practice on Learning , 2010 .

[61]  N. Finkelstein,et al.  Patterns of multiple representation use by experts and novices during physics problem solving , 2008 .

[62]  Daniel M. Herbert,et al.  Study Drawings in Architectural Design: Their Properties as a Graphic Medium , 1988 .

[63]  D. Clements,et al.  STUDENTS' UNDERSTANDING OF THREE-DIMENSIONAL RECTANGULAR ARRAYS OF CUBES , 1996 .

[64]  James A. Stewart,et al.  Student problem solving in high school genetics , 1983 .

[65]  Nancy J. Nersessian,et al.  Building Cognition: The Construction of Computational Representations for Scientific Discovery , 2015, Cogn. Sci..

[66]  Bernard Longden Genetics - Are There Inherent Learning Difficulties?. , 1982 .

[67]  Prajakt Pande,et al.  How do experts and novices navigate chemistry representations – an eye tracking investigation , 2015 .

[68]  Loretta L. Jones,et al.  Investigating Students' Ability to Transfer Ideas Learned from Molecular Animations of the Dissolution Process. , 2008 .

[69]  E. Hutchins The cultural ecosystem of human cognition , 2014 .

[70]  Andrew Izsák Representational Competence and Algebraic Modeling , 2011 .

[71]  V. Goel Sketches of thought , 1995 .

[72]  David Kirsh,et al.  Thinking with external representations , 2010, AI & SOCIETY.

[73]  Melanie M. Cooper,et al.  Does mechanistic thinking improve student success in organic chemistry , 2012 .

[74]  Joseph Krajcik,et al.  Influence of levels of information as presented by different technologies on students' understanding of acid, base, and ph concepts , 1994 .

[75]  Daniel L. Schwartz,et al.  Shuttling Between Depictive Models and Abstract Rules: Induction and Fallback , 1996, Cogn. Sci..

[76]  Thomas J. Greenbowe,et al.  What can we do about Sue: A case study of competence , 1986 .

[77]  David Kirsh,et al.  Adapting the Environment Instead of Oneself , 2022 .

[78]  Bette Davidowitz,et al.  Student-generated submicro diagrams: a useful tool for teaching and learning chemical equations and stoichiometry , 2010 .

[79]  Marcia C. Linn,et al.  Using Interactive Technology to Support Students’ Understanding of the Greenhouse Effect and Global Warming , 2012 .

[80]  P. Viviani,et al.  Biological movements look uniform: evidence of motor-perceptual interactions. , 1992, Journal of experimental psychology. Human perception and performance.

[81]  Robert B. Kozma,et al.  Students Becoming Chemists: Developing Representationl Competence , 2005 .

[82]  Sanjay Chandrasekharan,et al.  Building to Discover: A Common Coding Model , 2009, Cogn. Sci..

[83]  L. Schauble,et al.  Cultivating Model-Based Reasoning in Science Education , 2005 .

[84]  M. Hegarty Mechanical reasoning by mental simulation , 2004, Trends in Cognitive Sciences.

[85]  Roy D. Pea,et al.  Distributed by Design: On the Promises and Pitfalls of Collaborative Learning with Multiple Representations , 2011 .

[86]  Fernando Hitt,et al.  Difficulties in the articulation of different representations linked to the concept of function , 1998 .

[87]  S. Anastasiadou,et al.  The role of representations in the understanding of probabilities in tertiary education , 2008 .

[88]  L. Schauble,et al.  Design Experiments in Educational Research , 2003 .

[89]  R. Kozma,et al.  Multimedia and understanding: Expert and novice responses to different representations of chemical phenomena , 1997 .

[90]  Jan T. van der Veen,et al.  The learning effects of computer simulations in science education , 2012, Comput. Educ..

[91]  Daniel L. Schwartz,et al.  Analog Imagery in Mental Model Reasoning: Depictive Models , 1996, Cognitive Psychology.

[92]  Orkun S. Soyer,et al.  The roles of integration in molecular systems biology. , 2012, Studies in history and philosophy of biological and biomedical sciences.

[93]  W. L. Yarroch Student Understanding of Chemical Equation Balancing. , 1985 .

[94]  Chi-Yan Tsui,et al.  Introduction to Multiple Representations: Their Importance in Biology and Biological Education , 2013 .

[95]  Mary B. Nakhleh,et al.  Students’ Microscopic, Macroscopic, and Symbolic Representations of Chemical Reactions , 1999 .

[96]  John S. Gero,et al.  Drawings and the design process , 1998 .

[97]  Wallace Feurzeig,et al.  Modeling and Simulation in Science and Mathematics Education , 1999, Modeling Dynamic Systems.

[98]  Mathematics teacher candidates skills of using multiple representations for division of fractions , 2014 .

[99]  David C. Gooding,et al.  From Phenomenology to Field Theory: Faraday's Visual Reasoning , 2006, Perspectives on Science.

[100]  Dor Abrahamson,et al.  Learning Is Moving in New Ways: The Ecological Dynamics of Mathematics Education , 2016 .

[101]  Cynthia J. Atman,et al.  Mapping between design activities and external representations for engineering student designers , 2006 .

[102]  Konrad J. Schönborn,et al.  A Model of Factors Determining Students’ Ability to Interpret External Representations in Biochemistry , 2009 .

[103]  Russell Tytler,et al.  Constructing Representations to Learn Science , 2016 .

[104]  C. Hirsch,et al.  Student preferences for representations of functions , 1992 .

[105]  Paul P. Maglio,et al.  On Distinguishing Epistemic from Pragmatic Action , 1994, Cogn. Sci..

[106]  Susan C. Nurrenbern,et al.  Concept Learning versus Problem Solving: Is There a Difference?. , 1987 .

[107]  Barton Moffatt,et al.  Distributed Cognition: , 2003 .

[108]  K. E. Shubbar Learning the Visualisation of Rotations in Diagrams of Three Dimensional Structures. , 1990 .

[109]  Russell Tytler,et al.  Reasoning in Science through Representation , 2013 .

[110]  Sahana Murthy,et al.  The Enactive equation: Exploring How Multiple External Representations are Integrated, Using a Fully Controllable Interface and Eye-Tracking , 2014, 2014 IEEE Sixth International Conference on Technology for Education.

[111]  Martin Reisslein,et al.  Technological Literacy Learning With Cumulative and Stepwise Integration of Equations Into Electrical Circuit Diagrams , 2012, IEEE Transactions on Education.

[112]  Mike Thomas,et al.  What is the object of the encapsulation of a process , 1999 .

[113]  Paul J. Feltovich,et al.  Categorization and Representation of Physics Problems by Experts and Novices , 1981, Cogn. Sci..

[114]  Kenneth R. Koedinger,et al.  Trade-Offs Between Grounded and Abstract Representations: Evidence From Algebra Problem Solving , 2008, Cogn. Sci..

[115]  Edwin Hutchins,et al.  How a Cockpit Remembers Its Speeds , 1995, Cogn. Sci..

[116]  G. Salomon Distributed cognitions : psychological and educational considerations , 1997 .

[117]  Liliana Mammino,et al.  Teaching Chemistry with and Without External Representations in Professional Environments with Limited Resources , 2008 .

[118]  Tommy Dreyfus,et al.  Advanced Mathematical Thinking Processes , 2002 .

[119]  Uri Wilensky,et al.  Crossing Levels and Representations: The Connected Chemistry (CC1) Curriculum , 2009 .

[120]  Mike Stieff,et al.  Increasing Representational Fluency with Visualization Tools , 2006, ICLS.

[121]  D. Treagust,et al.  Students’ Learning Strategies With Multiple Representations: Explanations of the Human Breathing Mechanism , 2014 .

[122]  Graphical Representations of Speed: Obstacles Preservice K-8 Teachers Experience. , 2000 .

[123]  Raymond Duval A Cognitive Analysis of Problems of Comprehension in a Learning of Mathematics , 2006 .

[124]  Andrea A. diSessa,et al.  Meta-representation: an introduction , 2000 .

[125]  Wolfgang Schnotz,et al.  Commentary: Towards an Integrated View of Learning from Text and Visual Displays , 2002 .

[126]  Paul Marshall,et al.  Do tangible interfaces enhance learning? , 2007, TEI.

[127]  Nancy J. Nersessian,et al.  Creating Scientific Concepts , 2008 .

[128]  Mike Stieff,et al.  Identifying Representational Competence With Multi-Representational Displays , 2011 .

[129]  George M. Bodner,et al.  Using students' representations constructed during problem solving to infer conceptual understanding , 2012 .

[130]  James H. Stewart,et al.  Difficulties Experienced by High School Students when Learning Basic Mendelian Genetics. , 1982 .

[131]  F. M. Garforth Ionic Equations--Difficulties in Understanding and Use. , 1976 .

[132]  Michael J. Sanger Evaluating Students' Conceptual Understanding of Balanced Equations and Stoichiometric Ratios Using a Particulate Drawing , 2005 .

[133]  G. Goldin,et al.  A Joint Perspective on the Idea of Representation in Learning and Doing Mathematics , 2013 .

[134]  Mike Stieff,et al.  Connected Chemistry—Incorporating Interactive Simulations into the Chemistry Classroom , 2003 .

[135]  Herbert A. Simon,et al.  Why a Diagram is (Sometimes) Worth Ten Thousand Words , 1987, Cogn. Sci..

[136]  Manuel Santos-Trigo,et al.  An exploration of strategies used by students to solve problems with multiple ways of solution , 1996 .

[137]  Frances R. Curcio,et al.  The Roles of Representation in School Mathematics: 2001 Yearbook , 2001 .

[138]  John K. Gilbert,et al.  Visualization in science education , 2005 .

[139]  Anna Sfard,et al.  Problems of Reification: Representations and Mathematical Objects , 1994 .

[140]  Yuichiro Anzai,et al.  Internal Models in Physics Problem Solving , 1984 .

[141]  James D. Hollan,et al.  Distributed cognition: toward a new foundation for human-computer interaction research , 2000, TCHI.

[142]  R. Kozma,et al.  The Roles of Representations and Tools in the Chemistry Laboratory and Their Implications for Chemistry Learning , 2000 .

[143]  Pratim Sengupta,et al.  Development of Mechanistic Reasoning and Multilevel Explanations of Ecology in Third Grade Using Agent-Based Models. , 2016 .

[144]  V. Prain,et al.  Learning Through Constructing Representations in Science: A framework of representational construction affordances , 2012 .

[145]  MICHAEL J. JACOBSON,et al.  Problem solving, cognition, and complex systems: Differences between experts and novices , 2001, Complex..

[146]  Aditya Johri,et al.  The Role of Representations in Engineering Practices: Taking a Turn towards Inscriptions , 2013 .

[147]  Iliada Elia,et al.  Relations Between Secondary Pupils’ Conceptions About Functions and Problem Solving in Different Representations , 2007 .

[148]  Shaaron Ainsworth,et al.  The functions of multiple representations , 1999, Comput. Educ..

[149]  Phillip J. Grimaldi,et al.  Uncovering the connection between artist and audience: Viewing painted brushstrokes evokes corresponding action representations in the observer , 2012, Cognition.

[150]  Kenneth Tobin,et al.  Cascades of inscriptions and the re-presentation of nature: how numbers, tables, graphs, and money come to re-present a rolling ball , 1997 .

[151]  L. Verschaffel,et al.  What counts as a flexible representational choice? An evaluation of students’ representational choices to solve linear function problems , 2012 .

[152]  David Kirsh,et al.  The Context of Work , 2001, Hum. Comput. Interact..

[153]  L. Verschaffel,et al.  Representational flexibility in linear-function problems: a choice/no-choice study , 2010 .

[154]  Micah S. Stohlmann,et al.  Modeling in Engineering: The Role of Representational Fluency in Students' Conceptual Understanding , 2013 .

[155]  P. Viviani,et al.  The effect of movement velocity on form perception: Geometric illusions in dynamic displays , 1989, Perception & psychophysics.

[156]  Dorothea P. Simon,et al.  Expert and Novice Performance in Solving Physics Problems , 1980, Science.

[157]  R. Kozma The material features of multiple representations and their cognitive and social affordances for science understanding , 2003 .

[158]  Daniel L. Schwartz,et al.  Physically Distributed Learning: Adapting and Reinterpreting Physical Environments in the Development of Fraction Concepts , 2005, Cogn. Sci..

[159]  E. Wiebe,et al.  The influence of prior knowledge on viewing and interpreting graphics with macroscopic and molecular representations , 2008 .

[160]  John K. Gilbert,et al.  Visualization: A Metacognitive Skill in Science and Science Education , 2005 .