Epistemic Network Analyses of Economics Students’ Graph Understanding: An Eye-Tracking Study

Learning to solve graph tasks is one of the key prerequisites of acquiring domain-specific knowledge in most study domains. Analyses of graph understanding often use eye-tracking and focus on analyzing how much time students spend gazing at particular areas of a graph—Areas of Interest (AOIs). To gain a deeper insight into students’ task-solving process, we argue that the gaze shifts between students’ fixations on different AOIs (so-termed transitions) also need to be included in holistic analyses of graph understanding that consider the importance of transitions for the task-solving process. Thus, we introduced Epistemic Network Analysis (ENA) as a novel approach to analyze eye-tracking data of 23 university students who solved eight multiple-choice graph tasks in physics and economics. ENA is a method for quantifying, visualizing, and interpreting network data allowing a weighted analysis of the gaze patterns of both correct and incorrect graph task solvers considering the interrelations between fixations and transitions. After an analysis of the differences in the number of fixations and the number of single transitions between correct and incorrect solvers, we conducted an ENA for each task. We demonstrate that an isolated analysis of fixations and transitions provides only a limited insight into graph solving behavior. In contrast, ENA identifies differences between the gaze patterns of students who solved the graph tasks correctly and incorrectly across the multiple graph tasks. For instance, incorrect solvers shifted their gaze from the graph to the x-axis and from the question to the graph comparatively more often than correct solvers. The results indicate that incorrect solvers often have problems transferring textual information into graphical information and rely more on partly irrelevant parts of a graph. Finally, we discuss how the findings can be used to design experimental studies and for innovative instructional procedures in higher education.

[1]  David Williamson Shaffer,et al.  A Tutorial on Epistemic Network Analysis: Analyzing the Structure of Connections in Cognitive, Social, and Interaction Data , 2016, J. Learn. Anal..

[2]  David Rosengrant,et al.  Comparing Experts and Novices in Solving Electrical Circuit Problems with the Help of Eye‐Tracking , 2009 .

[3]  A. Renkl,et al.  How multiple external representations are used and how they can be made more useful , 2009 .

[4]  S. Brückner,et al.  Validating a Test for Measuring Knowledge and Understanding of Economics Among University Students , 2019, Zeitschrift für Pädagogische Psychologie.

[5]  Tomasz Opach,et al.  Breaking the Eyes: How Do Users Get Started with a Coordinated and Multiple View Geovisualization Tool? , 2020 .

[6]  Sean Andrist,et al.  Look together: analyzing gaze coordination with epistemic network analysis , 2015, Front. Psychol..

[7]  Wolff-Michael Roth,et al.  Lecturing graphing: What features of lectures contribute to student difficulties in learning to interpret graph? , 1998 .

[8]  S. Ainsworth.  DeFT: A Conceptual Framework for Considering Learning with Multiple Representations. , 2006 .

[9]  Anne Schüler.  Investigating gaze behavior during processing of inconsistent text-picture information: Evidence for text-picture integration , 2017 .

[10]  Tesa Stegner,et al.  Which Students Benefit from Graphs in a Principles of Economics Class? , 2003 .

[11]  W. Schnotz,et al.  Surface and deep structures in graphics comprehension , 2014, Memory & Cognition.

[12]  David Williamson Shaffer,et al.  The hands and head of a surgeon: Modeling operative competency with multimodal epistemic network analysis. , 2017, American journal of surgery.

[13]  Huei-Tse Hou,et al.  Visual attention for solving multiple-choice science problem: An eye-tracking analysis , 2012, Comput. Educ..

[14]  Jochen Kuhn,et al.  Visual cues improve students’ understanding of divergence and curl: Evidence from eye movements during reading and problem solving , 2019, Physical Review Physics Education Research.

[15]  E. Cohn,et al.  Do Graphs Promote Learning in Principles of Economics? , 2001 .

[16]  Hermann G. Ebner,et al.  Improving cross-content transfer in text processing by means of active graphical representation , 2003 .

[17]  S. Küchemann,et al.  Student understanding of graph slope and area under a curve: A replication study comparing first-year physics and economics students , 2019, Physical Review Physics Education Research.

[18]  Yu-Cin Jian,et al.  Learners' eye movements during construction of mechanical kinematic representations from static diagrams , 2014 .

[19]  David Williamson Shaffer,et al.  Using epistemic network analysis to identify targets for educational interventions in trauma team communication , 2018, Surgery.

[20]  A. Sušac,et al.  Comparison of university students’ understanding of graphs in different contexts , 2013 .

[21]  Jacob L. Orquin,et al.  Attention and choice: a review on eye movements in decision making. , 2013, Acta psychologica.

[22]  Michael Burch,et al.  Visualization of Eye Tracking Data: A Taxonomy and Survey , 2017, Comput. Graph. Forum.

[23]  The Impact of Entry Preconditions on Student Dropout and Subject Change in Business and Economics , 2020 .

[24]  Jose P. Mestre,et al.  Eye-Gaze Patterns as Students Study Worked-out Examples in Mechanics. , 2010 .

[25]  Kenneth Holmqvist,et al.  Eye tracking: a comprehensive guide to methods and measures , 2011 .

[26]  Benjamin Strobel,et al.  Task-irrelevant data impair processing of graph reading tasks: An eye tracking study , 2018, Learning and Instruction.

[27]  Joseph H. Goldberg,et al.  Identifying fixations and saccades in eye-tracking protocols , 2000, ETRA.

[28]  Marlit Annalena Lindner,et al.  Test-takers' eye movements: Effects of integration aids and types of graphical representations , 2017, Comput. Educ..

[29]  Taiga Brahm,et al.  Do difficulty levels matter for graphical literacy? A performance assessment study with authentic graphs , 2019, International Journal of Science Education.

[30]  Ivana Tichá,et al.  Impact of Think-Aloud on Eye-Tracking: A Comparison of Concurrent and Retrospective Think-Aloud for Research on Decision-Making in the Game Environment , 2020, Sensors.

[31]  Zachari Swiecki,et al.  In Search of Conversational Grain Size: Modeling Semantic Structure Using Moving Stanza Windows , 2017, ICLS.

[32]  Lillian C. McDermott,et al.  Student difficulties in connecting graphs and physics: Examples from kinematics , 1987 .

[33]  Priti Shah,et al.  Toward a Model of Knowledge-Based Graph Comprehension , 2002, Diagrams.

[34]  Jochen Kuhn,et al.  Förderung von Repräsentationskompetenz und Experimentbezug in den vorlesungsbegleitenden Übungen zur Experimentalphysik , 2018 .

[35]  Robert J. Beichner,et al.  Testing student interpretation of kinematics graphs , 1994 .

[36]  Gaea Leinhardt,et al.  Functions, Graphs, and Graphing: Tasks, Learning, and Teaching , 1990 .

[37]  Michael Schuhen,et al.  Informelles Lernen und ökonomische Bildung , 2016 .

[38]  M. Bannert,et al.  Construction and interference in learning from multiple representation , 2003 .

[39]  Yeliz Yesilada,et al.  Eye tracking scanpath analysis techniques on web pages: A survey, evaluation and comparison , 2015 .

[40]  Steven Pinker,et al.  A theory of graph comprehension. , 1990 .

[41]  Michael Burch,et al.  Visual Analytics Methodology for Eye Movement Studies , 2012, IEEE Transactions on Visualization and Computer Graphics.

[42]  Steven J. Landry,et al.  An Eye Movement Analysis Algorithm for a Multielement Target Tracking Task: Maximum Transition-Based Agglomerative Hierarchical Clustering , 2015, IEEE Transactions on Human-Machine Systems.

[43]  David Williamson Shaffer,et al.  Epistemic Network Analysis: A Worked Example of Theory-Based Learning Analytics , 2017 .

[44]  Lester C. Loschky,et al.  Differences in visual attention between those who correctly and incorrectly answer physics problems , 2012 .

[45]  Susan N. Friel,et al.  Making sense of graphs: Critical factors influencing comprehension and instructional implications. , 2001 .

[46]  Priti Shah,et al.  A Model of the Perceptual and Conceptual Processes in Graph Comprehension , 1998 .

[47]  Gary E. Raney,et al.  Using Eye Movements to Evaluate the Cognitive Processes Involved in Text Comprehension , 2014, Journal of visualized experiments : JoVE.

[48]  Ana Susac,et al.  Student understanding of graph slope and area under a graph: A comparison of physics and nonphysics students , 2018, Physical Review Physics Education Research.

[49]  S. Brückner,et al.  Changes in Students’ Understanding of and Visual Attention on Digitally Represented Graphs Across Two Domains in Higher Education: A Postreplication Study , 2020, Frontiers in Psychology.

[50]  Maria Caterina Tornatora,et al.  An Eye-Tracking Study of Learning From Science Text With Concrete and Abstract Illustrations , 2013 .

[51]  Bradley W. Bergey,et al.  Teaching High School Biology Students to Coordinate Text and Diagrams: Relations with Transfer, Effort, and Spatial Skill , 2015 .

[52]  Juliane Junker,et al.  Artificial Intelligence And The Future Of Testing , 2016 .

[53]  Frances R. Curcio,et al.  Comprehension of Mathematical Relationships Expressed in Graphs. , 1987 .

[54]  David Williamson Shaffer,et al.  Quantifying the qualitative with epistemic network analysis: A human factors case study of task-allocation communication in a primary care team , 2018, IISE transactions on healthcare systems engineering.

[55]  Olga Zlatkin-Troitschanskaia,et al.  Influences on the Development of Economic Knowledge over the First Academic Year , 2020 .

[56]  P. V. Kampen,et al.  Categorization of first-year university students' interpretations of numerical linear distance-time graphs , 2013 .