Uncovering the cognitive processes underlying mental rotation: an eye-movement study

Mental rotation is an important paradigm for spatial ability. Mental-rotation tasks are assumed to involve five or three sequential cognitive-processing states, though this has not been demonstrated experimentally. Here, we investigated how processing states alternate during mental-rotation tasks. Inference was carried out using an advanced statistical modelling and data-driven approach – a discriminative hidden Markov model (dHMM) trained using eye-movement data obtained from an experiment consisting of two different strategies: (I) mentally rotate the right-side figure to be aligned with the left-side figure and (II) mentally rotate the left-side figure to be aligned with the right-side figure. Eye movements were found to contain the necessary information for determining the processing strategy, and the dHMM that best fit our data segmented the mental-rotation process into three hidden states, which we termed encoding and searching, comparison, and searching on one-side pair. Additionally, we applied three classification methods, logistic regression, support vector model and dHMM, of which dHMM predicted the strategies with the highest accuracy (76.8%). Our study did confirm that there are differences in processing states between these two of mental-rotation strategies, and were consistent with the previous suggestion that mental rotation is discrete process that is accomplished in a piecemeal fashion.

[1]  John R. Anderson.  Arguments concerning representations for mental imagery. , 1978 .

[2]  John R. Anderson,et al.  Mapping eye movements to cognitive processes , 1999 .

[3]  Krzysztof Pancerz,et al.  Toward Synchronization of EEG and Eye-tracking Data Using an Expert System , 2015, CS&P.

[4]  A. Larsen.  Deconstructing mental rotation. , 2014, Journal of experimental psychology. Human perception and performance.

[5]  Eileen Kowler.  Eye movements and their role in visual and cognitive processes. , 1990, Reviews of oculomotor research.

[6]  Angelo Cangelosi,et al.  Generalisation, decision making, and embodiment effects in mental rotation: A neurorobotic architecture tested with a humanoid robot , 2015, Neural Networks.

[7]  Mark J. van der Laan,et al.  Fitting of mixtures with unspecified number of components using cross validation distance estimate , 2003, Comput. Stat. Data Anal..

[8]  P Suppes,et al.  Eye-movement models for arithmetic and reading performance. , 1990, Reviews of oculomotor research.

[9]  P. Sachse,et al.  Eye Movements during Mental Rotation of Nonmirrored and Mirrored Three-Dimensional Abstract Objects , 2011, Perceptual and motor skills.

[10]  D. Taylor.  Eye Movements and Psychological Processes , 1977 .

[11]  Victoria A Roach,et al.  The eye of the beholder: Can patterns in eye movement reveal aptitudes for spatial reasoning? , 2016, Anatomical sciences education.

[12]  Ilpo Kojo,et al.  Using hidden Markov model to uncover processing states from eye movements in information search tasks , 2008, Cognitive Systems Research.

[13]  Drew H. Abney,et al.  Journal of Experimental Psychology : Human Perception and Performance Influence of Musical Groove on Postural Sway , 2015 .

[14]  M. Just,et al.  Eye fixations and cognitive processes , 1976, Cognitive Psychology.

[15]  Silvia Wen-Yu Lee,et al.  A review of using eye-tracking technology in exploring learning from 2000 to 2012 , 2013 .

[16]  D. Ballard,et al.  What you see is what you need. , 2003, Journal of vision.

[17]  James J. Clark,et al.  Visual Task Inference Using Hidden Markov Models , 2011, IJCAI.

[18]  Hubert D Zimmer,et al.  Think spatial: the representation in mental rotation is nonvisual. , 2013, Journal of experimental psychology. Learning, memory, and cognition.

[19]  R. Groner.  Eye Movements and Psychological Functions: International Views , 1983 .

[20]  M. Peters,et al.  Applications of mental rotation figures of the Shepard and Metzler type and description of a mental rotation stimulus library , 2008, Brain and Cognition.

[21]  D. Metzler,et al.  Mental rotation: effects of dimensionality of objects and type of task. , 1988, Journal of experimental psychology. Human perception and performance.

[22]  Michael C. Corballis,et al.  Mental Rotation and the Right Hemisphere , 1997, Brain and Language.

[23]  M. Just,et al.  Cognitive coordinate systems: accounts of mental rotation and individual differences in spatial ability. , 1985, Psychological review.

[24]  R. Pieters,et al.  Visual Attention to Repeated Print Advertising: A Test of Scanpath Theory , 1999 .

[25]  L. Chalupa,et al.  The visual neurosciences , 2004 .

[26]  D. Whitteridge.  Movements of the eyes R. H. S. Carpenter, Pion Ltd, London (1977), 420 pp., $27.00 , 1979, Neuroscience.

[27]  R. C. Langford.  How People Look at Pictures, A Study of the Psychology of Perception in Art. , 1936 .

[28]  N. Thacker,et al.  Functional Specialisation in Human Premotor Cortex : VisuoSpatial Transformation in Pre-SMA During 2 D Image Transformation , 2009 .

[29]  Biing-Hwang Juang,et al.  Hidden Markov Models for Speech Recognition , 1991 .

[30]  R. Almond.  The therapeutic community. , 1971, Scientific American.

[31]  R. Shepard,et al.  CHRONOMETRIC STUDIES OF THE ROTATION OF MENTAL IMAGES , 1973 .

[32]  M. Corballis.  Recognition of disoriented shapes. , 1988, Psychological review.

[33]  R. Shepard,et al.  Mental Rotation of Three-Dimensional Objects , 1971, Science.

[34]  Tanja Neumann,et al.  Eye Movements And The Higher Psychological Functions , 2016 .

[35]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[36]  T. Wüstenberg,et al.  Mirrored or identical — Is the role of visual perception underestimated in the mental rotation process of 3D-objects?: A combined fMRI-eye tracking-study , 2012, Neuropsychologia.

[37]  D. Noton,et al.  Eye movements and visual perception. , 1971, Scientific American.

[38]  K. Rayner.  Eye movements in reading and information processing: 20 years of research. , 1998, Psychological bulletin.

[39]  Qiang Ji,et al.  In the Eye of the Beholder: A Survey of Models for Eyes and Gaze , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Z. Pylyshyn.  Return of the mental image: are there really pictures in the brain? , 2003, Trends in Cognitive Sciences.

[41]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[42]  A. L. Yarbus.  Eye Movements During Perception of Complex Objects , 1967 .

[43]  Eileen Kowler.  Eye movements: The past 25years , 2011, Vision Research.

[44]  J. H. Steiger,et al.  Nonholistic processing in mental rotation: Some suggestive evidence , 1982, Perception & psychophysics.

[45]  Thomas F. Shipley,et al.  Individual Differences in Mental Rotation : Piecemeal vs Holistic Processes , 2011, CogSci.

[46]  Zenon W. Pylyshyn,et al.  What the Mind’s Eye Tells the Mind’s Brain: A Critique of Mental Imagery , 1973 .

[47]  S. Tong,et al.  Spatiotemporal differences of brain activation between internal and external strategies in mental rotation: A behavioral and ERD/ERS study , 2016, Neuroscience Letters.

[48]  Vladimir Naumovich Vapni.  The Nature of Statistical Learning Theory , 1995 .

[49]  M. Hegarty,et al.  Individual differences in mental rotation: piecemeal versus holistic processing. , 2013, Experimental psychology.

[50]  R. Luce,et al.  Effects of stimulus complexity on mental rotation rate of polygons. , 1987, Journal of experimental psychology. Human perception and performance.

[51]  D. Ballard,et al.  Eye movements in natural behavior , 2005, Trends in Cognitive Sciences.

[52]  N. Newman.  The Visual Neurosciences , 2005 .