Sex difference in spatial ability for college students and exploration of measurement invariance

Abstract This study investigates psychometric evidence for a spatial ability instrument with a focus on measurement invariance and sex difference in this domain. The Purdue Visualization of Rotations (ROT) test was administered to science major students enrolled in a college chemistry course to measure mental rotation ability. Confirmatory factor analysis (CFA) was used to evaluate the fit of three alternative models. A bi-factor model is a better fit for the data than other models (1-factor, 4-factor 2nd-order), which indicates that the probability of answering a test item correctly is affected not only by student ability but also by an item-writing pattern. Results from multiple-group CFA support the measurement invariance of the bi-factor model between sexes and the outperformance of males over females with medium to large effect sizes. In conclusion, the ROT is appropriate to measure student mental rotation ability in college classrooms and to detect a difference between sexes.

[1]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[2]  C. Hill,et al.  Why So Few? Women in Science, Technology, Engineering, and Mathematics. , 2010 .

[3]  Geoffrey L. Cohen,et al.  Supporting Online Material for Reducing the Gender Achievement Gap in College Science: A Classroom Study of Values Affirmation , 2010 .

[4]  L. Cronbach,et al.  Construct validity in psychological tests. , 1955, Psychological bulletin.

[5]  S. West,et al.  A Comparison of Bifactor and Second-Order Models of Quality of Life , 2006, Multivariate behavioral research.

[6]  S. Brownlow,et al.  Science Background and Spatial Abilities in Men and Women , 2003 .

[7]  L. Angeles.  Evaluating Cutoff Criteria of Model Fit Indices for Latent Variable Models with Binary and Continuous Outcomes , 2002 .

[8]  James P. Stevens,et al.  Intermediate Statistics: A Modern Approach , 1990 .

[9]  D. Lubinski,et al.  Spatial ability for STEM domains: Aligning over 50 years of cumulative psychological knowledge solidifies its importance. , 2009 .

[10]  So Yoon Yoon,et al.  A Meta-Analysis on Gender Differences in Mental Rotation Ability Measured by the Purdue Spatial Visualization Tests: Visualization of Rotations (PSVT:R) , 2013 .

[11]  T. Raykov,et al.  Measurement Invariance for Latent Constructs in Multiple Populations , 2012 .

[12]  N. Newcombe,et al.  A longitudinal study of predictors of spatial ability in adolescent females. , 1992, Child development.

[13]  G. Mcclearn,et al.  Variability and stability in cognitive abilities are largely genetic later in life , 1994, Behavior genetics.

[14]  M. J. Contreras,et al.  On the Robustness of Solution Strategy Classifications: Testing the Stability of Dynamic Spatial Tasks on a One-Year Test-Retest Basis , 2010 .

[15]  K. Petrides,et al.  Ethnic, Gender, and Socio-Economic Group Differences in Academic Performance and Secondary School Selection: A Longitudinal Analysis. , 2008 .

[16]  Zahira Merchant,et al.  The learner characteristics, features of desktop 3D virtual reality environments, and college chemistry instruction: A structural equation modeling analysis , 2012, Comput. Educ..

[17]  P. Bentler,et al.  Comparative fit indexes in structural models. , 1990, Psychological bulletin.

[18]  R. Hoyle.  Structural equation modeling: concepts, issues, and applications , 1997 .

[19]  Lisa Marie Weckbacher,et al.  Mental rotation ability in relation to self-perceptions of high school geometry , 2014 .

[20]  M. Sommer,et al.  Evaluating the contribution of different item features to the effect size of the gender difference in three-dimensional mental rotation using automatic item generation , 2010 .

[21]  S. Vandenberg,et al.  Mental Rotations, a Group Test of Three-Dimensional Spatial Visualization , 1978, Perceptual and motor skills.

[22]  B. Tabachnick,et al.  Using Multivariate Statistics , 1983 .

[23]  S. Reise,et al.  Bifactor Models and Rotations: Exploring the Extent to Which Multidimensional Data Yield Univocal Scale Scores , 2010, Journal of personality assessment.

[24]  Michael T. Battista,et al.  The Importance of Spatial Visualization and Cognitive Development for Geometry Learning in Preservice Elementary Teachers. , 1982 .

[25]  Xiaoying Xu,et al.  Understanding the State of the Art for Measurement in Chemistry Education Research: Examining the Psychometric Evidence , 2013 .

[26]  Sheryl A. Sorby,et al.  A Course for the Development of 3-D Spatial Visualization Skills. , 1996 .

[27]  J. Lewis,et al.  Supporting Alternative Strategies for Learning Chemical Applications of Group Theory. , 2013 .

[28]  K. Murphy,et al.  Psychological Testing: Principles and Applications , 1988 .

[29]  Renee L. Babcock,et al.  Age and Experience Effects in Spatial Visualization , 1990 .

[30]  M. Towns,et al.  A Review of Spatial Ability Literature, Its Connection to Chemistry, and Implications for Instruction , 2011 .

[31]  Susan M. Barnett,et al.  Women's underrepresentation in science: sociocultural and biological considerations. , 2009, Psychological bulletin.

[32]  Donald Hedeker,et al.  Full-Information Item Bifactor Analysis of Graded Response Data , 2007 .

[33]  D. Bors,et al.  Sex Differences on the Mental Rotation Test: An Analysis of Item Types. , 2011 .

[34]  Frank Rijmen,et al.  Formal Relations and an Empirical Comparison among the Bi‐Factor, the Testlet, and a Second‐Order Multidimensional IRT Model , 2010 .

[35]  D. Lohman.  Spatial Ability: A Review and Reanalysis of the Correlational Literature. , 1979 .

[36]  Georg Gittler,et al.  On gender differences in mental rotation processing speed , 2014 .

[37]  P. Meyer.  Understanding Measurement: Reliability , 2010 .

[38]  D. Halpern,et al.  Can spatial training improve long-term outcomes for gifted STEM undergraduates? , 2013 .

[39]  Adam E. Christensen,et al.  Spatial Foundations of Science Education: The Illustrative Case of Instruction on Introductory Geological Concepts , 2011 .

[40]  Susan D. Voyer,et al.  Magnitude of sex differences in spatial abilities: a meta-analysis and consideration of critical variables. , 1995, Psychological bulletin.

[41]  J. Blickenstaff.  Women and science careers: leaky pipeline or gender filter? , 2005 .

[42]  J. Lewis,et al.  The application and evaluation of a two-concept diagnostic instrument with students entering college general chemistry , 2012 .

[43]  José M. Tomás,et al.  Rosenberg's Self-Esteem Scale: Two Factors or Method Effects. , 1999 .

[44]  G. Hancock.  Effect size, power, and sample size determination for structured means modeling and mimic approaches to between-groups hypothesis testing of means on a single latent construct , 2001 .

[45]  K. Markon,et al.  The reliability and validity of discrete and continuous measures of psychopathology: a quantitative review. , 2011, Psychological bulletin.

[46]  R. Shepard,et al.  Mental Rotation of Three-Dimensional Objects , 1971, Science.

[47]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[48]  P. Bentler,et al.  Evaluating model fit. , 1995 .

[49]  S. Hamstra,et al.  What counts as validity evidence? Examples and prevalence in a systematic review of simulation-based assessment , 2014, Advances in health sciences education : theory and practice.

[50]  H. Holling,et al.  Retest effects in matrix test performance: Differential impact of predictors at different hierarchy levels in an educational setting ☆ , 2011 .

[51]  R. Shepard,et al.  CHRONOMETRIC STUDIES OF THE ROTATION OF MENTAL IMAGES , 1973 .

[52]  L. A. Tartre.  Spatial Orientation Skill and Mathematical Problem Solving. , 1990 .

[53]  Howard T. Everson,et al.  Methodology Review: Statistical Approaches for Assessing Measurement Bias , 1993 .

[54]  Alice A. Black.  Spatial Ability and Earth Science Conceptual Understanding , 2005 .

[55]  E. S. Kim,et al.  Testing Measurement Invariance: A Comparison of Multiple-Group Categorical CFA and IRT , 2011 .

[56]  Julia Dmitrieva,et al.  Item-wording and the dimensionality of the Rosenberg Self-Esteem Scale: do they matter? , 2003 .

[57]  S. Reise,et al.  Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. , 1997 .

[58]  K. Schaie,et al.  Structural Invariance of Cognitive Abilities across the Adult Life Span: A Cross-Sectional Study. , 1989 .

[59]  Janet Shibley Hyde,et al.  Cross-national patterns of gender differences in mathematics: a meta-analysis. , 2010, Psychological bulletin.

[60]  Brian A. Nosek,et al.  National differences in gender–science stereotypes predict national sex differences in science and math achievement , 2009, Proceedings of the National Academy of Sciences.

[61]  Mary A. LaRussa,et al.  A STUDY OF TWO MEASURES OF SPATIAL ABILITY AS PREDICTORS OF SUCCESS IN DIFFERENT LEVELS OF GENERAL CHEMISTRY , 1987 .

[62]  M. Towns.  Developing Learning Objectives and Assessment Plans at a Variety of Institutions: Examples and Case Studies , 2010 .

[63]  George M. Bodner,et al.  Spatial Ability and its Role in Organic Chemistry: A Study of Four Organic Courses. , 1987 .

[64]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[65]  M. G. McGee.  Human spatial abilities: psychometric studies and environmental, genetic, hormonal, and neurological influences. , 1979, Psychological bulletin.

[66]  Christine E. DeMars.  A Tutorial on Interpreting Bifactor Model Scores , 2013 .

[67]  Sheryl A. Sorby,et al.  Developing 3-D Spatial Visualization Skills. , 1999 .

[68]  T. Lord.  Enhancing the visuo‐spatial aptitude of students , 1985 .

[69]  GEORGE M. BODNER,et al.  The Purdue Visualization of Rotations Test , 1997 .

[70]  M. Linn,et al.  Emergence and characterization of sex differences in spatial ability: a meta-analysis. , 1985, Child development.

[71]  Inci Morgil,et al.  Traditional and computer-assisted learning in teaching acids and bases , 2005 .

[72]  So Yoon Yoon,et al.  Psychometric properties of the Revised Purdue Spatial Visualization Tests: Visualization of Rotations (the Revised PSVT:R) , 2011 .

[73]  M. Stieff.  Mental rotation and diagrammatic reasoning in science , 2007 .

[74]  Alija Kulenović,et al.  Standards for Educational and Psychological Testing , 1999 .