User Evaluation of Map-Based Visual Analytic Tools

Big data have also become a big challenge for cartographers, as the majority of big data may be localized. The use of visual analytics tools, as well as comprising interactive maps, stimulates inter-disciplinary actors to explore new ideas and decision-making methods. This paper deals with the evaluation of three map-based visual analytics tools by means of the eye-tracking method. The conceptual part of the paper begins with an analysis of the state-of-the-art and ends with the design of proof-of-concept experiments. The verification part consists of the design, composition, and realization of the conducted eye-tracking experiment, in which three map-based visual analytics tools were tested in terms of user-friendliness. A set of recommendations on GUI (graphical user interface) design and interactive functionality for map makers is formulated on the basis of the discovered errors and shortcomings in the assessed stimuli. The results of the verification were used as inputs for improving the three tested map-based visual analytics tools and might serve as a best practice for map-based visual analytics tools in general, as well as for improving the policy making cycle as elaborated by the European project PoliVisu (Policy Development based on Advanced Geospatial Data Analytics and Visualization).

[1]  Karel Jedlicka,et al.  Open transport map — Routable OpenStreetMap , 2016, 2016 IST-Africa Week Conference.

[2]  Jaroslav Burian.  SPATIAL ASPECTS OF MOVEMENT OF OLOMOUC AND OSTRAVA CITIZENS , 2016 .

[3]  Arthur M. Jacobs,et al.  OGAMA (Open Gaze and Mouse Analyzer): Open-source software designed to analyze eye and mouse movements in slideshow study designs , 2008, Behavior research methods.

[4]  Hong Zhou,et al.  Visual Clustering in Parallel Coordinates , 2008, Comput. Graph. Forum.

[5]  Philippe Cerfontaine,et al.  How virtual reality can help visualise and assess geohazards , 2019, Int. J. Digit. Earth.

[6]  Matthew Adams,et al.  Usability of MyFireWatch for non-expert users measured by eyetracking , 2018 .

[7]  J.C. Roberts,et al.  State of the Art: Coordinated & Multiple Views in Exploratory Visualization , 2007, Fifth International Conference on Coordinated and Multiple Views in Exploratory Visualization (CMV 2007).

[8]  Arzu Çöltekin,et al.  The Next Generation of Atlas User Interfaces: A User Study with "Digital Natives" , 2014, CARTOCON.

[9]  Jean Scholtz,et al.  Beyond Usability: Evaluation Aspects of Visual Analytic Environments , 2006, 2006 IEEE Symposium On Visual Analytics Science And Technology.

[10]  Daniel A. Keim,et al.  Geovisual analytics for spatial decision support: Setting the research agenda , 2007, Int. J. Geogr. Inf. Sci..

[11]  Lukás Herman,et al.  Evaluation of User Performance in Interactive and Static 3D Maps , 2018, ISPRS Int. J. Geo Inf..

[12]  Susie Ruston McAleer,et al.  The Case for Collaborative Policy Experimentation Using Advanced Geospatial Data Analytics and Visualisation , 2017, INSCI Workshops.

[13]  Michael Batty,et al.  Virtual Environments Begin to Embrace Process‐based Geographic Analysis , 2015, Trans. GIS.

[14]  Jin Chen,et al.  Combining Usability Techniques to Design Geovisualization Tools for Epidemiology , 2005, Cartography and geographic information science.

[15]  Petr Kubíček,et al.  Prototyping the visualization of geographic and sensor data for agriculture , 2013 .

[16]  Melanie Tory,et al.  Evaluating Visualizations: Do Expert Reviews Work? , 2005, IEEE Computer Graphics and Applications.

[17]  Patton Sawicki Clark,et al.  Basic methods of policy analysis and planning , 1986 .

[18]  Arzu Çöltekin,et al.  Exploring the efficiency of users' visual analytics strategies based on sequence analysis of eye movement recordings , 2010, Int. J. Geogr. Inf. Sci..

[19]  Jimmy Johansson,et al.  Map-Based Web Tools Supporting Climate Change Adaptation , 2015 .

[20]  Karel Charvát,et al.  Open Farm Management Information System Supporting Ecological and Economical Tasks , 2017, ISESS.

[21]  Lukáš Herman,et al.  Eye-tracking Analysis of Interactive 3D Geovisualization , 2017, Journal of eye movement research.

[22]  Mark Harrower,et al.  Addressing Map Interface Usability: Learning from the Lakeshore Nature Preserve Interactive Map , 2008 .

[23]  Rostislav Netek,et al.  Implementation of Heat Maps in Geographical Information System – Exploratory Study on Traffic Accident Data , 2018 .

[24]  Stanislav Popelka,et al.  Evaluation of the Cartographical Quality of Urban Plans by Eye-Tracking , 2018, ISPRS Int. J. Geo Inf..

[25]  Zdenek Stachon,et al.  Visualization Problems in Worldwide Map Portals , 2014, CARTOCON.

[26]  James R. Lewis,et al.  Evaluation of Procedures for Adjusting Problem-Discovery Rates Estimated From Small Samples , 2001, Int. J. Hum. Comput. Interact..

[27]  Philippe De Maeyer,et al.  Combining user logging with eye tracking for interactive and dynamic applications , 2015, Behavior research methods.

[28]  James R. Lewis.  Testing Small System Customer Set-Up , 1982 .

[29]  Kristin A. Cook,et al.  Illuminating the Path: The Research and Development Agenda for Visual Analytics , 2005 .

[30]  Lukáš Herman,et al.  3D Web Visualization of Environmental Information – Integration of Heterogeneous Data Sources when Providing Navigation and Interaction , 2015 .

[31]  Robert M. Edsall.  The parallel coordinate plot in action: design and use for geographic visualization , 2003, Comput. Stat. Data Anal..

[32]  Lukáš Herman,et al.  Population distribution modelling at fine spatio-temporal scale based on mobile phone data , 2018, Int. J. Digit. Earth.

[33]  Tomasz Opach,et al.  For your eyes only? Evaluating a coordinated and multiple views tool with a map, a parallel coordinated plot and a table using an eye-tracking approach , 2017, Int. J. Geogr. Inf. Sci..

[34]  Oleg V. Komogortsev,et al.  Qualitative and quantitative scoring and evaluation of the eye movement classification algorithms , 2010, ETRA.

[35]  Jean Scholtz,et al.  Evaluation of visual analytics environments: The road to the Visual Analytics Science and Technology challenge evaluation methodology , 2014, Inf. Vis..

[36]  Daniel A. Keim,et al.  Challenging problems of geospatial visual analytics , 2011, J. Vis. Lang. Comput..

[37]  Stanislav Popelka.  OPTIMAL EYE FIXATION DETECTION SETTINGS FOR CARTOGRAPHIC PURPOSES , 2014 .

[38]  Kristien Ooms,et al.  Measuring the Spatial Noise of a Low-Cost Eye Tracker to Enhance Fixation Detection , 2018, J. Imaging.

[39]  Jerry Shannon,et al.  Using Eye-tracking and Mouse Metrics to Test Usability of Web Mapping Navigation , 2012 .

[40]  Arzu Çöltekin,et al.  Evaluating the Effectiveness of Interactive Map Interface Designs: A Case Study Integrating Usability Metrics with Eye-Movement Analysis , 2009 .

[41]  Marco Winckler,et al.  On evaluating information visualization techniques , 2002, AVI '02.

[42]  Liangfeng Zhu,et al.  Representing Time-Dynamic Geospatial Objects on Virtual Globes Using CZML - Part I: Overview and Key Issues , 2018, ISPRS Int. J. Geo Inf..

[43]  Alex Singleton,et al.  The Rise of Big Spatial Data , 2017 .

[44]  P. Kiefer,et al.  FeaturEyeTrack: automatic matching of eye tracking data with map features on interactive maps , 2019, GeoInformatica.

[45]  Paweł Cybulski,et al.  A review on eye movement analysis in map reading process: the status of the last decade , 2023, Geodesy and Cartography.

[46]  Alena Vondráková,et al.  Eye-tracking Evaluation of Weather Web Maps , 2019, ISPRS Int. J. Geo Inf..

[47]  Jean Scholtz.  Developing qualitative metrics for visual analytic environments , 2010, BELIV '10.

[48]  P. Blignaut.  Fixation identification: The optimum threshold for a dispersion algorithm , 2009, Attention, perception & psychophysics.

[49]  Alan M. MacEachren,et al.  User-Centered Design for Interactive Maps: A Case Study in Crime Analysis , 2015, ISPRS Int. J. Geo Inf..

[50]  Michael Burch,et al.  Eye tracking evaluation of visual analytics , 2016, Inf. Vis..

[51]  Martin Raubal,et al.  Eye tracking for spatial research: Cognition, computation, challenges , 2017, Spatial Cogn. Comput..

[52]  Adam Mertel,et al.  Visualization and spatial analysis of police open data as a part of community policing in the city of Pardubice (Czech Republic) , 2016, Ann. GIS.

[53]  Igor Ivan,et al.  Sparse Big Data Problem. A Case Study of Czech Graffiti Crimes , 2017 .

[54]  Stanislav Popelka,et al.  ScanGraph: A Novel Scanpath Comparison Method Using Visualisation of Graph Cliques , 2016 .

[55]  Özge Alaçam,et al.  A Usability Study of WebMaps with Eye Tracking Tool: The Effects of Iconic Representation of Information , 2009, HCI.

[56]  Arzu Çöltekin,et al.  User studies in cartography: opportunities for empirical research on interactive maps and visualizations , 2017 .

[57]  Lukáš Herman,et al.  Visualization of Traffic Offences in the City of Brno (Czech Republic): Achieving 3D Thematic Cartography through Open Source and Open Data , 2018 .

[58]  Tomáš Mildorf,et al.  Design and Evaluation of WebGL-Based Heat Map Visualization for Big Point Data , 2017 .

[59]  Milan Konecný,et al.  Review: Cartography: challenges and potential in the virtual geographic environments era , 2011, Ann. GIS.