What Went Wrong for Bad Solvers during Thematic Map Analysis? Lessons Learned from an Eye-Tracking Study

Thematic map analysis is a complex and challenging task that might result in map user failure for many reasons. In the study reported here, we wanted to search for differences between successful and unsuccessful map users, focusing—unlike many similar studies—on strategies applied by users who give incorrect answers. In the eye-tracking study, followed by a questionnaire survey, we collected data from 39 participants. The eye-tracking data were analyzed both qualitatively and quantitatively to compare participants’ strategies from various perspectives. Unlike the results of some other studies, it turned out that unsuccessful participants show some similarities that are consistent across most analyzed tasks. The main issues that characterize bad solvers relate to improper use of the thematic legend, the inability to focus on relevant map layout elements, as well as on adequate map content. Moreover, they differed in the general problem-solving approach used as they, for example, tended to choose fast, less cautious, strategies. Based on the collected results, we developed tips that could help prevent unsuccessful participants ending with an incorrect answer and therefore be beneficial in map use education.

[1]  Roman Bednarik,et al.  Expertise-dependent visual attention strategies develop over time during debugging with multiple code representations , 2012, Int. J. Hum. Comput. Stud..

[2]  Tad T. Brunyé,et al.  When goals constrain: Eye movements and memory for goal‐Oriented map study , 2009 .

[3]  William J. Gerace.  Problem Solving and Conceptual Understanding , 2001 .

[4]  K. A. Ericsson,et al.  Protocol Analysis: Verbal Reports as Data , 1984 .

[5]  Bernhard Jenny,et al.  User preferences for world map projections , 2015 .

[6]  Arzu Çöltekin,et al.  Perceptual complexity of soil-landscape maps: a user evaluation of color organization in legend designs using eye tracking , 2017, Int. J. Digit. Earth.

[7]  F. Paas,et al.  Uncovering the problem-solving process: cued retrospective reporting versus concurrent and retrospective reporting. , 2005, Journal of experimental psychology. Applied.

[8]  Weihua Dong,et al.  Using Eye Tracking to Evaluate the Usability of Flow Maps , 2018, ISPRS Int. J. Geo Inf..

[9]  Carmel M. Diezmann,et al.  Co-thought gestures: Supporting students to successfully navigate map tasks , 2014 .

[10]  Senay Yasar,et al.  University Students’ Conceptualization and Interpretation of Topographic Maps , 2008 .

[11]  Kathryn A. Stofer,et al.  Comparing Experts and Novices on Scaffolded Data Visualizations using Eye-tracking , 2014 .

[12]  Rostislav Netek,et al.  Implementation of Heat Maps in Geographical Information System – Exploratory Study on Traffic Accident Data , 2018 .

[13]  Stanislav Popelka,et al.  Evaluation of the Cartographical Quality of Urban Plans by Eye-Tracking , 2018, ISPRS Int. J. Geo Inf..

[14]  Vladimir I. Levenshtein,et al.  Binary codes capable of correcting deletions, insertions, and reversals , 1965 .

[15]  Kostas Dimopoulos,et al.  Science on the Web: Secondary School Students’ Navigation Patterns and Preferred Pages’ Characteristics , 2010 .

[16]  Yeliz Yesilada,et al.  Eye tracking scanpath analysis techniques on web pages: A survey, evaluation and comparison , 2015 .

[17]  Izabela Gołębiowska,et al.  Legend Layouts for Thematic Maps: A Case Study Integrating Usability Metrics with the Thinking Aloud Method , 2015 .

[18]  Clayton Lewis,et al.  A problem-oriented classification of visualization techniques , 1990, Proceedings of the First IEEE Conference on Visualization: Visualization `90.

[19]  Tsu-Chiang Lei,et al.  Evaluating differences in spatial visual attention in wayfinding strategy when using 2D and 3D electronic maps , 2014, GeoJournal.

[20]  Stanislav Popelka,et al.  ScanGraph: A Novel Scanpath Comparison Method Using Visualisation of Graph Cliques , 2016 .

[21]  Frank Witlox,et al.  Interpreting maps through the eyes of expert and novice users , 2012, Int. J. Geogr. Inf. Sci..

[22]  J. Libarkin,et al.  Spatial Signatures of Mapping Expertise Among Field Geologists , 2012 .

[23]  Yolanda Postigo,et al.  The Learning of a Geographical Map by Experts and Novices , 1998 .

[24]  Seok-hee Cho.  The Role of IQ in the Use of Cognitive Strategies to Learn Information from a Map. , 2010 .

[25]  Arzu Çöltekin,et al.  Evaluating the Effectiveness of Interactive Map Interface Designs: A Case Study Integrating Usability Metrics with Eye-Movement Analysis , 2009 .

[26]  Jan-Louis Kruger,et al.  Attention distribution and cognitive load in a subtitled academic lecture: L1 vs. L2 , 2014 .

[27]  Benjamin Naumann.  The Architecture Of Cognition , 2016 .

[28]  Raymond Kent.  Analysing Quantitative Data: Variable-based and Case-based Approaches to Non-experimental Datasets , 2015 .

[29]  Hsiao-Ching She,et al.  Web-based undergraduate chemistry problem-solving: The interplay of task performance, domain knowledge and web-searching strategies , 2012, Comput. Educ..

[30]  Alan Kingstone,et al.  A comparison of scanpath comparison methods , 2014, Behavior Research Methods.

[31]  Alena Vondráková,et al.  Eye-tracking Evaluation of Weather Web Maps , 2019, ISPRS Int. J. Geo Inf..

[32]  R. Mayer.  Cognitive, metacognitive, and motivational aspects of problem solving , 1998 .

[33]  Lorenzo Fiorina,et al.  Thinking style, browsing primes and hypermedia navigation , 2007, Comput. Educ..

[34]  R. Nason,et al.  Modelling expertise in map reading: Beginnings , 1992 .

[35]  Peter A. Frensch,et al.  The Role of Information Reduction in Skill Acquisition , 1996, Cognitive Psychology.

[36]  T. Gog,et al.  In the eyes of the beholder: How experts and novices interpret dynamic stimuli , 2010 .

[37]  Lenka Havelková,et al.  Map Skills in Education: A Systematic Review of Terminology, Methodology, and Influencing Factors , 2019, Review of International Geographical Education Online.

[38]  R. Säljö,et al.  Expertise Differences in the Comprehension of Visualizations: a Meta-Analysis of Eye-Tracking Research in Professional Domains , 2011 .

[39]  Arzu Çöltekin,et al.  Exploring the efficiency of users' visual analytics strategies based on sequence analysis of eye movement recordings , 2010, Int. J. Geogr. Inf. Sci..

[40]  Cathleen Wharton,et al.  Cognitive Walkthroughs: A Method for Theory-Based Evaluation of User Interfaces , 1992, Int. J. Man Mach. Stud..

[41]  N. Fernandez,et al.  Quantifying Novice and Expert Differences in Visual Diagnostic Reasoning in Veterinary Pathology Using Eye-Tracking Technology. , 2018, Journal of veterinary medical education.

[42]  Weihua Dong,et al.  Assessing Map-Reading Skills Using Eye Tracking and Bayesian Structural Equation Modelling , 2018, Sustainability.

[43]  Tomasz Opach,et al.  For your eyes only? Evaluating a coordinated and multiple views tool with a map, a parallel coordinated plot and a table using an eye-tracking approach , 2017, Int. J. Geogr. Inf. Sci..

[44]  S. B. Needleman,et al.  A General Method Applicable to the Search for Similarities in the Amino Acid Sequence of Two Proteins , 1989 .

[45]  Patrik Pluchino,et al.  Do fourth graders integrate text and picture in processing and learning from an illustrated science text? Evidence from eye-movement patterns , 2013, Comput. Educ..

[46]  Wen-Chin Li,et al.  The Investigation of Visual Attention and Workload by Experts and Novices in the Cockpit , 2013, HCI.

[47]  Andrew T. Duchowski,et al.  Eye Tracking Methodology: Theory and Practice , 2003, Springer London.

[48]  Jeremy W. Crampton,et al.  A Cognitive Analysis of Wayfinding Expertise , 1992 .

[49]  Stanislav Popelka,et al.  EVALUATION OF THE USER STRATEGY ON 2D AND 3D CITY MAPS BASED ON NOVEL SCANPATH COMPARISON METHOD AND GRAPH VISUALIZATION , 2016 .

[50]  Jan Theeuwes,et al.  ScanMatch: A novel method for comparing fixation sequences , 2010, Behavior research methods.

[51]  Philippe De Maeyer,et al.  Analyzing eye movement patterns to improve map design , 2010 .

[52]  Stanislav Popelka.  OPTIMAL EYE FIXATION DETECTION SETTINGS FOR CARTOGRAPHIC PURPOSES , 2014 .

[53]  Veerle Fack,et al.  Listen to the Map User: Cognition, Memory, and Expertise , 2015 .

[54]  O. Svenson.  Differentiation and consolidation theory of human decision making: A frame of reference for the study of pre- and post-decision processes , 1992 .

[55]  L. S. Liben,et al.  Children's strategies and difficulties while using a map to record locations in an outdoor environment , 2010 .

[56]  Panos Markopoulos,et al.  A comparison of think-aloud, questionnaires and interviews for testing usability with children , 2002 .

[57]  Sara Irina Fabrikant,et al.  How Do People View Multi-Component Animated Maps? , 2014 .

[58]  Weihua Dong,et al.  Exploring differences of visual attention in pedestrian navigation when using 2D maps and 3D geo-browsers , 2017 .

[59]  Robert E. Roth,et al.  Cartographic Interaction Primitives: Framework and Synthesis , 2012 .

[60]  Lenka Havelková,et al.  Research into map-analysis strategies: theory- and data-driven approaches , 2019, Geografie.

[61]  Evanthia C. Michaelidou,et al.  The Ability of Elementary School Children to Analyse General Reference and Thematic Maps , 2004, Cartogr. Int. J. Geogr. Inf. Geovisualization.

[62]  Kenneth Holmqvist,et al.  All good readers are the same, but every low-skilled reader is different: an eye-tracking study using PISA data , 2018 .

[63]  Siliang Tang,et al.  Using Eye Tracking to Explore the Impacts of Geography Courses on Map-based Spatial Ability , 2018, Sustainability.

[64]  Petr Kubícek,et al.  Navigation in Indoor Environments: Does the Type of Visual Learning Stimulus Matter? , 2019, ISPRS Int. J. Geo Inf..

[65]  M. Just,et al.  Eye fixations and cognitive processes , 1976, Cognitive Psychology.

[66]  Martin Hanus,et al.  The Impact of Map Type on the Level of Student Map Skills , 2018, Cartogr. Int. J. Geogr. Inf. Geovisualization.

[67]  Removed.  Interpreting Map Usage Patterns using Geovisual Analytics and Spatio-Temporal Clustering , 2022 .

[68]  Arthur M. Jacobs,et al.  OGAMA (Open Gaze and Mouse Analyzer): Open-source software designed to analyze eye and mouse movements in slideshow study designs , 2008, Behavior research methods.

[69]  P. Walsh.  The Power of Maps. , 1998 .

[70]  Fang Xu,et al.  Using eye tracking to evaluate the usability of animated maps , 2013, Science China Earth Sciences.

[71]  Lester C. Loschky,et al.  How Does Visual Attention Differ Between Experts and Novices on Physics Problems , 2010 .

[73]  Philippe De Maeyer,et al.  EEG & Eye Tracking User Experiments for Spatial Memory Task on Maps , 2019, ISPRS Int. J. Geo Inf..

[74]  Alzbeta Brychtova,et al.  An Empirical User Study for Measuring the Influence of Colour Distance and Font Size in Map Reading Using Eye Tracking , 2016 .

[75]  Alan M. MacEachren,et al.  How Maps Work - Representation, Visualization, and Design , 1995 .

[76]  P. Maeyer,et al.  Study of the attentive behavior of novice and expert map users using eye tracking , 2014 .

[77]  Tsia-ying Hsieh,et al.  The Influence of Gender Difference on the Information-seeking Behaviors for the Graphical Interface of Children's Digital Library , 2015 .

[78]  F. Paas,et al.  Cognitive Load Theory and Instructional Design: Recent Developments , 2003 .

[79]  Weihua Dong,et al.  Using Eye Tracking to Explore Differences in Map-Based Spatial Ability between Geographers and Non-Geographers , 2018, ISPRS Int. J. Geo Inf..

[80]  R. Bednarz,et al.  Video Analysis of Map-drawing Strategies , 2005 .

[81]  James R. Eagan,et al.  Low-level components of analytic activity in information visualization , 2005, IEEE Symposium on Information Visualization, 2005. INFOVIS 2005..