A Review of Exploratory Factor Analysis Decisions and Overview of Current Practices: What We Are Doing and How Can We Improve?

Authors within the fields of cyberpsychology and human−computer interaction have demonstrated a particular interest in measurement and scale creation, and exploratory factor analysis (EFA) is an extremely important statistical method for these areas of research. Unfortunately, EFA requires several statistical and methodological decisions to which the best choices are often unclear. The current article reviews five primary decisions and provides direct suggestions for best practices. These decisions are (a) the data inspection techniques, (b) the factor analytic method, (c) the factor retention method, (d) the factor rotation method, and (e) the factor loading cutoff. Then the article reviews authors’ choices for these five EFA decisions in every relevant article within seven cyberpsychology and/or human–computer interaction journals. The results demonstrate that authors do not employ the recommended best practices for most decisions. Particularly, most authors do not inspect their data for violations of assumptions, apply inappropriate factor analytic methods, utilize outdated factor retention methods, and omit the justification for their factor rotation methods. Further, many authors omit altogether their EFA decisions. To rectify these concerns, the current article provides a step-by-step guide and checklist that authors can reference to ensure the use of recommended best practices. Together, the current article identifies concerns with current research and provides direct solutions to these concerns.

[1]  Jason W. Osborne,et al.  Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. , 2005 .

[2]  Dimitra Dodou,et al.  Factor recovery by principal axis factoring and maximum likelihood factor analysis as a function of factor pattern and sample size , 2012 .

[3]  James C. Hayton,et al.  Factor Retention Decisions in Exploratory Factor Analysis: a Tutorial on Parallel Analysis , 2004 .

[4]  Keith Zoski,et al.  An Objective Counterpart to the Visual Scree Test for Factor Analysis: The Standard Error Scree , 1996 .

[5]  S. Tobias,et al.  BRIEF REPORT: BARTLETT'S TEST OF SPHERICITY AND CHANCE FINDINGS IN FACTOR ANALYSIS. , 1969, Multivariate behavioral research.

[6]  Timothy R. Hinkin,et al.  A Brief Tutorial on the Development of Measures for Use in Survey Questionnaires , 1998 .

[7]  C. Dziuban,et al.  When is a correlation matrix appropriate for factor analysis? Some decision rules. , 1974 .

[8]  B. Price.  A First Course in Factor Analysis , 1993 .

[9]  Johan van Braak,et al.  The computer attributes for learning scale (CALS) among university students: Scale development and relationship with actual computer use for learning , 2007, Comput. Hum. Behav..

[10]  R. Michael Furr,et al.  Scale Construction and Psychometrics for Social and Personality Psychology , 2011 .

[11]  Sevgi Özkan,et al.  Multi-dimensional students' evaluation of e-learning systems in the higher education context: An empirical investigation , 2009, Comput. Educ..

[12]  Hatice Ferhan Odabasi,et al.  Unethical computer using behavior scale: A study of reliability and validity on Turkish university students , 2007, Comput. Educ..

[13]  Duane T. Wegener,et al.  Evaluating the use of exploratory factor analysis in psychological research. , 1999 .

[14]  Matt C. Howard,et al.  An Analysis of More Than 1, 400 Articles, 900 Scales, and 17 Years of Research: The State of Scales in Cyberpsychology, Behavior, and Social Networking , 2015, Cyberpsychology Behav. Soc. Netw..

[15]  Diana Adler,et al.  Using Multivariate Statistics , 2016 .

[16]  Richard L. Gorsuch.  Exploratory Factor Analysis , 1988 .

[17]  Edward E. Cureton,et al.  The weighted varimax rotation and the promax rotation , 1975 .

[18]  H. Kaiser.  A second generation little jiffy , 1970 .

[19]  James R. Baker,et al.  Creation and Validation of the Personal Blogging Style Scale , 2011, Cyberpsychology Behav. Soc. Netw..

[20]  Raymond B. Cattell,et al.  Handbook of multivariate experimental psychology , 1968 .

[21]  Charles D. Dziuban,et al.  On the Extraction of Components and the Applicability of the Factor Model , 1973 .

[22]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[23]  B. Tabachnick,et al.  Using multivariate statistics, 5th ed. , 2007 .

[24]  Ulla K. Bunz,et al.  The Computer-Email-Web (CEW) Fluency Scale--Development and Validation , 2001 .

[25]  M. Bartlett.  TESTS OF SIGNIFICANCE IN FACTOR ANALYSIS , 1950 .

[26]  Paul Golder,et al.  The Guttman-Kaiser Criterion as a Predictor of the Number of Common Factors , 1982 .

[27]  Stewart Smith,et al.  The Second Generation , 1988 .

[28]  W. Velicer,et al.  Comparison of five rules for determining the number of components to retain. , 1986 .

[29]  Yongxin Li,et al.  The validation of the E-Victimisation Scale (E-VS) and the E-Bullying Scale (E-BS) for adolescents , 2013, Comput. Hum. Behav..

[30]  Karl F. MacDorman,et al.  The Aesthetic Dimensions of U.S. and South Korean Responses to Web Home Pages: A Cross-Cultural Comparison , 2011, Int. J. Hum. Comput. Interact..

[31]  J. Horn.  A rationale and test for the number of factors in factor analysis , 1965, Psychometrika.

[32]  Paul Kline,et al.  An easy guide to factor analysis , 1993 .

[33]  H. Kaiser.  The Application of Electronic Computers to Factor Analysis , 1960 .

[34]  Juho Hamari,et al.  Measuring flow in gamification: Dispositional Flow Scale-2 , 2014, Comput. Hum. Behav..

[35]  R. Gorsuch.  Exploratory factor analysis: its role in item analysis. , 1997, Journal of personality assessment.

[36]  Matthias Brand,et al.  Validation and psychometric properties of a short version of Young's Internet Addiction Test , 2013, Comput. Hum. Behav..

[37]  Sanjay Mishra,et al.  Parallel Analysis Engine to Aid Determining Number of Factors to Retain (Computer software) , 2007 .

[38]  R. Henson,et al.  Use of Exploratory Factor Analysis in Published Research , 2006 .

[39]  Tao Yang,et al.  DEEP: Design-Oriented Evaluation of Perceived Usability , 2012, Int. J. Hum. Comput. Interact..

[40]  Sanjay Mishra,et al.  Efficient theory development and factor retention criteria: Abandon the ‘eigenvalue greater than one’ criterion , 2008 .

[41]  B P O'Connor,et al.  SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test , 2000, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[42]  Scott E. Caplan.  Problematic Internet use and psychosocial well-being: development of a theory-based cognitive-behavioral measurement instrument , 2002, Comput. Hum. Behav..

[43]  R. MacCallum,et al.  Sample size in factor analysis. , 1999 .

[44]  Gavriel Salvendy,et al.  Measuring Player Immersion in the Computer Game Narrative , 2009, Int. J. Hum. Comput. Interact..

[45]  H. Kaiser.  The varimax criterion for analytic rotation in factor analysis , 1958 .

[46]  Aspasia Togia,et al.  Computer anxiety and attitudes among undergraduate students in Greece , 2010, Comput. Hum. Behav..

[47]  Norman Cliff,et al.  The eigenvalues-greater-than-one rule and the reliability of components. , 1988 .

[48]  M. Browne.  An Overview of Analytic Rotation in Exploratory Factor Analysis , 2001 .

[49]  R. Jennrich,et al.  Rotation for simple loadings , 1966, Psychometrika.

[50]  Y Kano,et al.  On the Equivalence of Factors and Components. , 1990, Multivariate behavioral research.

[51]  Erkan Yaman,et al.  Cyber victim and bullying scale: A study of validity and reliability , 2011, Comput. Educ..

[52]  W. Velicer,et al.  Affects of variable and subject sampling on factor pattern recovery. , 1998 .

[53]  Matt C. Howard.  Creation of a Computer Self-Efficacy Measure: Analysis of Internal Consistency, Psychometric Properties, and Validity , 2014, Cyberpsychology Behav. Soc. Netw..

[54]  R. MacCallum,et al.  THE APPLICATION OF EXPLORATORY FACTOR ANALYSIS IN APPLIED PSYCHOLOGY: A CRITICAL REVIEW AND ANALYSIS , 1986 .

[55]  Marc D. Miller,et al.  An assessment of the psychometric properties of the computer attitude scale , 1996 .

[56]  Allen I. Huffcutt,et al.  A Review and Evaluation of Exploratory Factor Analysis Practices in Organizational Research , 2003 .

[57]  John P. Charlton,et al.  The Development and Validation of the Computer Apathy and Anxiety Scale , 1995 .

[58]  Kristopher J Preacher,et al.  Repairing Tom Swift's Electric Factor Analysis Machine , 2003 .