Applied Psychometrics: Writing-Up a Factor Analysis Construct Validation Study with Examples

Factor analysis is carried out to psychometrically evaluate measurement instruments with multiple items like questionnaires or ability tests. EFA and CFA are widely used in measurement applications for construct validation and scale refinement. One of the more critical aspects of any CFA or EFA is communicating results. This work described reporting essentials of EFA with goodness of fit indices and CFA research when they are used to validate a measurement instrument with continuous variables in a different population from the one originally created. An overview of the minimum information to be reported is included along with short extracts from real published reports. For each reported section basic information to be included is described along with an example-extract adapted from published factor analysis construct validation studies. Additional issues covered include: Cross-validation, Measurement Invariance across Age and Gender, Reliability (α and ω), AVE-Based Validity, Convergent and Discriminant Validity with Correlation Analysis and Normative Data. Properly reported EFA and CFA could contribute to the improvement of the quality of the measurement instruments. A summary of good practices in CFA and SEM reporting based on literature is also included.

[1]  James B. Schreiber,et al.  Reporting Structural Equation Modeling and Confirmatory Factor Analysis Results: A Review , 2006 .

[2]  Timothy A. Brown,et al.  Confirmatory factor analysis. , 2012 .

[3]  W. Velicer.  Determining the number of components from the matrix of partial correlations , 1976 .

[4]  B. Tabachnick,et al.  Using Multivariate Statistics , 1983 .

[5]  Leandre R. Fabrigar,et al.  Exploratory Factor Analysis , 2011 .

[6]  Chu Kim-prieto,et al.  New Well-being Measures: Short Scales to Assess Flourishing and Positive and Negative Feelings , 2010 .

[7]  B. Thompson.  Ten commandments of structural equation modeling. , 2000 .

[8]  K. Mardia.  Measures of multivariate skewness and kurtosis with applications , 1970 .

[9]  Bruce W. Smith,et al.  The brief resilience scale: Assessing the ability to bounce back , 2008, International journal of behavioral medicine.

[10]  M. Bartlett,et al.  A note on the multiplying factors for various chi square approximations , 1954 .

[11]  Gordon W. Cheung,et al.  Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance , 2002 .

[12]  Evan Mayo-Wilson,et al.  Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board task force report. , 2018, The American psychologist.

[13]  B. Byrne,et al.  Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. , 1989 .

[14]  R. MacCallum,et al.  Applications of structural equation modeling in psychological research. , 2000, Annual review of psychology.

[15]  A. Panter,et al.  Writing about structural equation models. , 1995 .

[16]  R. P. McDonald,et al.  Principles and practice in reporting structural equation analyses. , 2002, Psychological methods.

[17]  F. Floyd,et al.  Factor analysis in the development and refinement of clinical assessment instruments. , 1995 .

[18]  T. Kubiak,et al.  Population-based validation of a German version of the Brief Resilience Scale , 2018, PloS one.

[19]  A. Stalikas,et al.  Using Bifactor EFA, Bifactor CFA and Exploratory Structural Equation Modeling to Validate Factor Structure of the Meaning in Life Questionnaire, Greek Version , 2018 .

[20]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[21]  E. Kevin Kelloway,et al.  Using Mplus for Structural Equation Modeling: A Researcher's Guide , 2014 .

[22]  The Fallibility of Our Judgments , 1992 .

[23]  Shigehiro Oishi,et al.  The Meaning in Life Questionnaire: Assessing the Presence of and Search for Meaning in Life , 2006 .

[24]  A. Stalikas,et al.  Can the Depression Anxiety Stress Scales Short Be Shorter? Factor Structure and Measurement Invariance of DASS-21 and DASS-9 in a Greek, Non-Clinical Sample , 2018 .

[25]  Bruce Thompson,et al.  The pivotal role of replication in psychological research , 1994 .

[26]  Urbano Lorenzo-Seva,et al.  The Hull Method for Selecting the Number of Common Factors , 2011, Multivariate behavioral research.

[27]  D. Lawley.  VI.—The Estimation of Factor Loadings by the Method of Maximum Likelihood , 1940 .

[28]  David F. Larcker,et al.  Structural Equation Models with Unobservable Variables and Measurement Error: Algebra and Statistics: , 1981 .

[29]  Theodoros A. Kyriazos,et al.  Psychometric Evidence of the Brief Resilience Scale (BRS) and Modeling Distinctiveness of Resilience from Depression and Stress , 2018 .

[30]  Theodoros A. Kyriazos,et al.  Is the Greek Version of Teacher’s Evaluation of Student’s Conduct (TESC) a Valid and Reliable Measure? , 2018 .

[31]  C. Hill,et al.  A Guide to Conducting Consensual Qualitative Research , 1997 .

[32]  K. Bollen.  A New Incremental Fit Index for General Structural Equation Models , 1989 .

[33]  Theodore A. Walls,et al.  Non-Graphical Solutions for Cattell’s Scree Test , 2013 .

[34]  J. S. Long,et al.  Tests for Structural Equation Models , 1992 .

[35]  Rick H Hoyle,et al.  Reporting Results from Structural Equation Modeling Analyses in Archives of Scientific Psychology. , 2013, Archives of scientific psychology.

[36]  H. Kaiser.  An index of factorial simplicity , 1974 .

[37]  Theodoros A. Kyriazos,et al.  A 3-Faced Construct Validation and a Bifactor Subjective Well-Being Model Using the Scale of Positive and Negative Experience, Greek Version , 2018 .

[38]  Kamlesh Singh,et al.  Validating the Flourishing Scale and the Scale of Positive and Negative Experience in India , 2016, Assessment of Mental Health, Religion and Culture.

[39]  A. Panter,et al.  The structural equation modeling research report. , 2012 .

[40]  Anthony G. Greenwald,et al.  Applications of Covariance Structure Modeling in Psychology : Cause for Concern ? , 2001 .

[41]  S. J. Sinclair,et al.  Psychometric Evaluation and Normative Data for the Depression, Anxiety, and Stress Scales-21 (DASS-21) in a Nonclinical Sample of U.S. Adults , 2012, Evaluation & the health professions.

[42]  H. Kaiser.  A second generation little jiffy , 1970 .

[43]  K. Jöreskog,et al.  Intraclass Reliability Estimates: Testing Structural Assumptions , 1974 .

[44]  M. Bartlett.  THE EFFECT OF STANDARDIZATION ON A χ2 APPROXIMATION IN FACTOR ANALYSIS , 1951 .

[45]  Brian A. Nosek,et al.  Recommendations for Increasing Replicability in Psychology † , 2013 .

[46]  Theodoros Kyriazos.  Applied Psychometrics: The 3-Faced Construct Validation Method, a Routine for Evaluating a Factor Structure , 2018 .

[47]  J. Horn.  A rationale and test for the number of factors in factor analysis , 1965, Psychometrika.

[48]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[49]  Peter M. Bentler,et al.  EQS : structural equations program manual , 1989 .

[50]  Todd D. Little,et al.  Common Fallacies in Quantitative Research Methodology , 2013 .

[51]  James B Schreiber,et al.  Core reporting practices in structural equation modeling. , 2008, Research in social & administrative pharmacy : RSAP.

[52]  J. Creswell,et al.  Journal article reporting standards for qualitative primary, qualitative meta-analytic, and mixed methods research in psychology: The APA Publications and Communications Board task force report. , 2018, The American psychologist.

[53]  Theodoros A. Kyriazos,et al.  The Flow Short Scale (FSS) Dimensionality and What MIMIC Shows on Heterogeneity and Invariance , 2018 .

[54]  T. Brown.  Confirmatory factor analysis for applied research, 2nd ed. , 2015 .

[55]  J. H. Steiger.  Aspects of Person-Machine Communication in Structural Modeling of Correlations and Covariances. , 1988, Multivariate behavioral research.

[56]  Robert F. DeVellis,et al.  Scale Development: Theory and Applications. , 1992 .

[57]  A. Stalikas,et al.  Validation of the Flourishing Scale (FS), Greek Version and Evaluation of Two Well-Being Models , 2018 .

[58]  R. Cattell.  The Scree Test For The Number Of Factors. , 1966, Multivariate behavioral research.

[59]  A. Boomsma.  Reporting Analyses of Covariance Structures , 2000 .

[60]  J R Nesselroade,et al.  Reporting structural equation modeling results in Psychology and Aging: some proposed guidelines. , 1991, Psychology and aging.

[61]  W Revelle,et al.  Very Simple Structure: An Alternative Procedure For Estimating The Optimal Number Of Interpretable Factors. , 1979, Multivariate behavioral research.

[62]  R. Darlington,et al.  Factor Analysis , 2008 .