Time limits in testing: An analysis of eye movements and visual attention in spatial problem solving

Individuals with an aptitude for interpreting spatial information (high mental rotation ability: HMRA) typically master anatomy with more ease, and more quickly, than those with low mental rotation ability (LMRA). This article explores how visual attention differs with time limits on spatial reasoning tests. Participants were assorted to two groups based on their mental rotation ability scores and their eye movements were collected during these tests. Analysis of salience during testing revealed similarities between MRA groups in untimed conditions but significant differences between the groups in the timed one. Question‐by‐question analyses demonstrate that HMRA individuals were more consistent across the two timing conditions (κ = 0.25), than the LMRA (κ = 0.013). It is clear that the groups respond to time limits differently and their apprehension of images during spatial problem solving differs significantly. Without time restrictions, salience analysis suggests LMRA individuals attended to similar aspects of the images as HMRA and their test scores rose concomitantly. Under timed conditions however, LMRA diverge from HMRA attention patterns, adopting inflexible approaches to visual search and attaining lower test scores. With this in mind, anatomical educators may wish to revisit some evaluations and teaching approaches in their own practice. Although examinations need to evaluate understanding of anatomical relationships, the addition of time limits may induce an unforeseen interaction of spatial reasoning and anatomical knowledge. Anat Sci Educ 10: 528–537. © 2017 American Association of Anatomists.

[1]  D. Schnipke.  Assessing Speededness in Computer-Based Tests Using Item Response Times , 1995 .

[2]  Silvia Wen-Yu Lee,et al.  A review of using eye-tracking technology in exploring learning from 2000 to 2012 , 2013 .

[3]  D. Lohman.  Spatial Ability: A Review and Reanalysis of the Correlational Literature. , 1979 .

[4]  Claudia Mello-Thoms,et al.  Time course of perception and decision making during mammographic interpretation. , 2002, AJR. American journal of roentgenology.

[5]  Timothy D Wilson,et al.  Visuospatial anatomy comprehension: The role of spatial visualization ability and problem‐solving strategies , 2014, Anatomical sciences education.

[6]  Carla A. Romney,et al.  Effect of visual–spatial ability on medical students' performance in a gross anatomy course , 2012, Anatomical sciences education.

[7]  R. Fisher.  On the Interpretation of χ2 from Contingency Tables, and the Calculation of P , 2010 .

[8]  Timothy D. Wilson,et al.  Computer visualizations: Factors that influence spatial anatomy comprehension , 2012, Anatomical sciences education.

[9]  N. Charness,et al.  The perceptual aspect of skilled performance in chess: Evidence from eye movements , 2001, Memory & cognition.

[10]  Ruth B. Ekstrom,et al.  Manual for kit of factor-referenced cognitive tests , 1976 .

[11]  M. Brandt,et al.  Visual-spatial ability, learning modality and surgical knot tying. , 2006, Canadian journal of surgery. Journal canadien de chirurgie.

[12]  R. Sternberg.  Toward a triarchic theory of human intelligence , 1984, Behavioral and Brain Sciences.

[13]  M. Peters,et al.  Applications of mental rotation figures of the Shepard and Metzler type and description of a mental rotation stimulus library , 2008, Brain and Cognition.

[14]  M. Just,et al.  Cognitive coordinate systems: accounts of mental rotation and individual differences in spatial ability. , 1985, Psychological review.

[15]  Xuezhu Ren,et al.  The position effect in tests with a time limit: the consideration of interruption and working speed , 2013 .

[16]  Randall W Engle,et al.  Working memory, short-term memory, and general fluid intelligence: a latent-variable approach. , 1999, Journal of experimental psychology. General.

[17]  Timothy D. Wilson,et al.  Role of Image and Cognitive Load in Anatomical Multimedia , 2020, Teaching Anatomy.

[18]  S. Vandenberg,et al.  Mental Rotations, a Group Test of Three-Dimensional Spatial Visualization , 1978, Perceptual and motor skills.

[19]  Mary Hegarty,et al.  Spatial Visualization in Physics Problem Solving , 2007, Cogn. Sci..

[20]  Michael J. Spivey,et al.  Eye Movements and Problem Solving , 2003, Psychological science.

[21]  A. Zumwalt,et al.  Gaze patterns of gross anatomy students change with classroom learning , 2015, Anatomical sciences education.

[22]  E. Krupinski,et al.  Visual scanning patterns of radiologists searching mammograms. , 1996, Academic radiology.

[23]  M. Peters,et al.  A Redrawn Vandenberg and Kuse Mental Rotations Test - Different Versions and Factors That Affect Performance , 1995, Brain and Cognition.

[24]  Arthur C. Graesser,et al.  Question asking and eye tracking during cognitive disequilibrium: Comprehending illustrated texts on devices when the devices break down , 2005, Memory & cognition.

[25]  Fisher'S Exact Test of Mutual Independence for 2 X 2 X 2 Cross-Classification Tables , 1994 .

[26]  C. Spearman.  The Abilities of Man their Nature and Measurement , 2020, Nature.

[27]  Patrick C. Kyllonen,et al.  Reasoning ability is (little more than) working-memory capacity?! , 1990 .

[28]  H L Kundel,et al.  A visual concept shapes image perception. , 1983, Radiology.

[29]  P. Chapman,et al.  Visual Search of Driving Situations: Danger and Experience , 1998, Perception.

[30]  Bob Rehder,et al.  Feature inference learning and eyetracking , 2009 .

[31]  Laura E. Thomas,et al.  Moving eyes and moving thought: On the spatial compatibility between eye movements and cognition , 2007, Psychonomic bulletin & review.

[32]  O. Wilhelm,et al.  The relation of speeded and unspeeded reasoning with mental speed , 2002 .

[33]  Victoria A Roach,et al.  The eye of the beholder: Can patterns in eye movement reveal aptitudes for spatial reasoning? , 2016, Anatomical sciences education.

[34]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[35]  M. Peters,et al.  Spatial Ability, Student Gender, and Academic Performance , 1995 .

[36]  D. C. Howell.  Statistical Methods for Psychology , 1987 .

[37]  Stephen G. Sireci,et al.  Validity Issues in Test Speededness , 2007 .

[38]  A. Miyake,et al.  The Cambridge Handbook of Visuospatial Thinking , 2005 .

[39]  Michael Eid,et al.  Separating "Rotators" From "Nonrotators" in the Mental Rotations Test: A Multigroup Latent Class Analysis , 2006, Multivariate behavioral research.

[40]  A. Viera,et al.  Understanding interobserver agreement: the kappa statistic. , 2005, Family medicine.

[41]  Keith K Niall,et al.  Mental Rotation, Pictured Rotation, and Tandem Rotation in Depth , 1997 .

[42]  M. Just,et al.  Eye fixations and cognitive processes , 1976, Cognitive Psychology.

[43]  O. Wilhelm,et al.  Working memory capacity - facets of a cognitive ability construct , 2000 .

[44]  Hsiao-Ching She,et al.  The impact of multimedia effect on science learning: Evidence from eye movements , 2009, Comput. Educ..

[45]  M. McHugh.  Interrater reliability: the kappa statistic , 2012, Biochemia medica.

[46]  Victoria A Roach,et al.  Different perspectives: Spatial ability influences where individuals look on a timed spatial test , 2017, Anatomical sciences education.

[47]  Homer H. Chen,et al.  Learning-Based Prediction of Visual Attention for Video Signals , 2011, IEEE Transactions on Image Processing.

[48]  Ethan D Grober,et al.  Visual-spatial ability correlates with efficiency of hand motion and successful surgical performance. , 2003, Surgery.

[49]  M. Hegarty,et al.  Individual Differences in Spatial Abilities , 2005 .

[50]  S. Hamstra,et al.  Effect of visual-spatial ability on learning of spatially-complex surgical skills , 2002, The Lancet.

[51]  F. Paas,et al.  Uncovering the problem-solving process: cued retrospective reporting versus concurrent and retrospective reporting. , 2005, Journal of experimental psychology. Applied.

[52]  Mark R. Wilson,et al.  Perceptual impairment and psychomotor control in virtual laparoscopic surgery , 2011, Surgical Endoscopy.

[53]  H. Haider,et al.  Eye movement during skill acquisition: More evidence for the information-reduction hypothesis , 1999 .

[54]  Marcus Nyström,et al.  Learning perceptual aspects of diagnosis in medicine via eye movement modeling examples on patient video cases , 2010 .

[55]  W. Kirchner.  Age differences in short-term retention of rapidly changing information. , 1958, Journal of experimental psychology.

[56]  Brent Bridgeman,et al.  Testing and Time Limits , 2004 .

[57]  David F. Lohman,et al.  Spatial abilities as traits, processes, and knowledge. , 1988 .

[58]  R. Mayer,et al.  Nine Ways to Reduce Cognitive Load in Multimedia Learning , 2003 .