Professional Differences: A Comparative Study of Visualization Task Performance and Spatial Ability Across Disciplines

Problem-driven visualization work is rooted in deeply understanding the data, actors, processes, and workflows of a target domain. However, an individual's personality traits and cognitive abilities may also influence visualization use. Diverse user needs and abilities raise natural questions for specificity in visualization design: Could individuals from different domains exhibit performance differences when using visualizations? Are any systematic variations related to their cognitive abilities? This study bridges domain-specific perspectives on visualization design with those provided by cognition and perception. We measure variations in visualization task performance across chemistry, computer science, and education, and relate these differences to variations in spatial ability. We conducted an online study with over 60 domain experts consisting of tasks related to pie charts, isocontour plots, and 3D scatterplots, and grounded by a well-documented spatial ability test. Task performance (correctness) varied with profession across more complex visualizations (isocontour plots and scatterplots), but not pie charts, a comparatively common visualization. We found that correctness correlates with spatial ability, and the professions differ in terms of spatial ability. These results indicate that domains differ not only in the specifics of their data and tasks, but also in terms of how effectively their constituent members engage with visualizations and their cognitive traits. Analyzing participants' confidence and strategy comments suggests that focusing on performance neglects important nuances, such as differing approaches to engage with even common visualizations and potential skill transference. Our findings offer a fresh perspective on discipline-specific visualization with specific recommendations to help guide visualization design that celebrates the uniqueness of the disciplines and individuals we seek to serve.

[1]  Kyle Wm. Hall,et al.  Design by Immersion: A Transdisciplinary Approach to Problem-Driven Visualizations , 2019, IEEE Transactions on Visualization and Computer Graphics.

[2]  Thomas Lowrie,et al.  Visuospatial training improves elementary students’ mathematics performance , 2017, The British journal of educational psychology.

[3]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[4]  R C Williges,et al.  Assaying and Isolating Individual Differences in Searching a Hierarchical File System , 1987, Human factors.

[5]  Mary Hegarty,et al.  Correction to: Decision making with visualizations: a cognitive framework across disciplines , 2018, Cognitive Research: Principles and Implications.

[6]  So Yoon Yoon,et al.  College students' performance in an introductory atmospheric science course: associations with spatial ability , 2016 .

[7]  Marilyn Tremaine,et al.  Understanding visualization through spatial ability differences , 2005, VIS 05. IEEE Visualization, 2005..

[8]  Lane Harrison,et al.  Towards a 3-dimensional model of individual cognitive differences: position paper , 2012, BELIV '12.

[9]  Mary Hegarty,et al.  Individual differences in use of external visualisations to perform an internal visualisation task. , 2007 .

[10]  D. Lubinski,et al.  Importance of Assessing Spatial Ability in Intellectually Talented Young Adolescents: A 20-Year Longitudinal Study. , 2001 .

[11]  M. Newman,et al.  A notation for the study of certain stereochemical problems , 1955 .

[12]  Robert Michael Kirby,et al.  Visualization Collaborations: What Works and Why , 2013, IEEE Computer Graphics and Applications.

[13]  So Yoon Yoon,et al.  Are Gender Differences in Spatial Ability Real or an Artifact? Evaluation of Measurement Invariance on the Revised PSVT:R , 2016 .

[14]  David Lubinski,et al.  Spatial Ability: A Neglected Dimension In Talent Searches For Intellectually Precocious Youth , 2007 .

[15]  P Beatty,et al.  Bridging the gaps. , 1990, Hospital trustee.

[16]  Heike Hofmann,et al.  Spatial Reasoning and Data Displays , 2016, IEEE Transactions on Visualization and Computer Graphics.

[17]  D. Lubinski,et al.  Spatial ability for STEM domains: Aligning over 50 years of cumulative psychological knowledge solidifies its importance. , 2009 .

[18]  So Yoon Yoon,et al.  A Meta-Analysis on Gender Differences in Mental Rotation Ability Measured by the Purdue Spatial Visualization Tests: Visualization of Rotations (PSVT:R) , 2013 .

[19]  Remco Chang,et al.  From Vision Science to Data Science: Applying Perception to Problems in Big Data , 2016, HVEI.

[20]  Steven Franconeri,et al.  The Curse of Knowledge in Visual Data Communication , 2019, IEEE Transactions on Visualization and Computer Graphics.

[21]  Pierre Dragicevic,et al.  Fair Statistical Communication in HCI , 2016 .

[22]  So Yoon Yoon,et al.  Psychometric properties of the Revised Purdue Spatial Visualization Tests: Visualization of Rotations (the Revised PSVT:R) , 2011 .

[23]  M. Stieff.  Mental rotation and diagrammatic reasoning in science , 2007 .

[24]  J. Hollas.  Modern Spectroscopy , 1936, Nature.

[25]  So Yoon Yoon,et al.  Exploring the Spatial Ability of Undergraduate Students: Association With Gender, STEM Majors, and Gifted Program Membership , 2017 .

[26]  R. J. Crouser,et al.  Survey on Individual Differences in Visualization , 2020, Comput. Graph. Forum.

[27]  Peter Bacchetti,et al.  Current sample size conventions: Flaws, harms, and alternatives , 2010, BMC medicine.

[28]  Lane Harrison,et al.  Data Visualization Literacy and Visualization Biases: Cases for Merging Parallel Threads , 2018, Cognitive Biases in Visualizations.

[29]  Martin Krzywinski,et al.  Points of Significance: Error bars , 2013, Nature Methods.

[30]  S. Kita,et al.  The nature of gestures ' beneficial role in spatial problem solving , 2013 .

[31]  Kim J. Vicente,et al.  Accommodating Individual Differences in Searching a Hierarchical File System , 1988, Int. J. Man Mach. Stud..

[32]  A. Roche,et al.  Organic Chemistry: , 1982, Nature.

[33]  Mike Stieff,et al.  Identifying Representational Competence With Multi-Representational Displays , 2011 .

[34]  E. L. Short,et al.  Quantum Chemistry , 1969, Nature.

[35]  S. Sorby,et al.  The role of spatial training in improving spatial and calculus performance in engineering students , 2013 .

[36]  Daniel A. Keim,et al.  Bridging the Gap of Domain and Visualization Experts with a Liaison , 2015, EuroVis.

[37]  J. Kendall.  Inorganic Chemistry , 1944, Nature.

[38]  Mary Czerwinski,et al.  Spatial ability and visual navigation: an Empirical Study , 1997, New Rev. Hypermedia Multim..

[39]  Wenhong Luo,et al.  User choice of interactive data visualization format: The effects of cognitive style and spatial ability , 2019, Decis. Support Syst..

[40]  James Agutter,et al.  Transactions on Visualization and Computer Graphics Design Activity Framework for Visualization Design , 2014 .

[41]  Kelly Caine,et al.  Local Standards for Sample Size at CHI , 2016, CHI.

[42]  Chaomei Chen.  Individual differences in a spatial-semantic virtual environment , 2000, J. Am. Soc. Inf. Sci..

[43]  D. Halpern,et al.  Can spatial training improve long-term outcomes for gifted STEM undergraduates? , 2013 .

[44]  G. Cumming.  Understanding the New Statistics: Effect Sizes, Confidence Intervals, and Meta-Analysis , 2011 .

[45]  Tamara Munzner,et al.  A Nested Model for Visualization Design and Validation , 2009, IEEE Transactions on Visualization and Computer Graphics.

[46]  Kelly S. Mix,et al.  Spatial Training Improves Children's Mathematics Ability , 2014 .

[47]  P. K. Imbrie,et al.  Psychometric properties of the revised PSVT:R for measuring first year engineering students’ spatial ability , 2013 .

[48]  Paul Parsons,et al.  Promoting Representational Fluency for Cognitive Bias Mitigation in Information Visualization , 2018, Cognitive Biases in Visualizations.

[49]  D. Lohman.  Spatial Ability: A Review and Reanalysis of the Correlational Literature. , 1979 .

[50]  Tobias Isenberg,et al.  The State of the Art of Spatial Interfaces for 3D Visualization , 2021, Comput. Graph. Forum.

[51]  Julianna Szendrei,et al.  Concrete Materials in the Classroom , 1996 .

[52]  R. Sundberg,et al.  Advanced Organic Chemistry: Part A: Structure and Mechanisms 5th Edition , 2020 .

[53]  Daniel Afergan,et al.  Improving Bayesian Reasoning: The Effects of Phrasing, Visualization, and Spatial Ability , 2016, IEEE Transactions on Visualization and Computer Graphics.

[54]  G. Schatz,et al.  Graphical Excellence. , 2014, The journal of physical chemistry letters.

[55]  Danielle Albers Szafir,et al.  Understanding Data Accessibility for People with Intellectual and Developmental Disabilities , 2021, CHI.

[56]  Micah S. Stohlmann,et al.  Modeling in Engineering: The Role of Representational Fluency in Students' Conceptual Understanding , 2013 .

[57]  Melanie Tory,et al.  Evaluation of Static and Dynamic Visualization Training Approaches for Users with Different Spatial Abilities , 2013, IEEE Transactions on Visualization and Computer Graphics.

[58]  Carl Gutwin,et al.  Threats of a replication crisis in empirical computer science , 2020, Commun. ACM.

[59]  石原 忍.  Tests for Colour-Blindness , 1910, Nature.

[60]  Tamara Munzner,et al.  Design Study Methodology: Reflections from the Trenches and the Stacks , 2012, IEEE Transactions on Visualization and Computer Graphics.

[61]  M. Towns,et al.  A Review of Spatial Ability Literature, Its Connection to Chemistry, and Implications for Instruction , 2011 .

[62]  M. Muir.  Physical Chemistry , 1888, Nature.

[63]  Huahai Yang,et al.  Personality as a Predictor of User Strategy: How Locus of Control Affects Search Strategies on Tree Visualizations , 2015, CHI.

[64]  D. Uttal,et al.  How Much Can Spatial Training Improve STEM Achievement? , 2015 .

[65]  Darren Edge,et al.  Beyond Tasks: An Activity Typology for Visual Analytics , 2018, IEEE Transactions on Visualization and Computer Graphics.