Psychological Methods Thanks Coefficient Alpha , We ’ ll Take It From Here

Empirical studies in psychology commonly report Cronbach’s alpha as a measure of internal consistency reliability despite the fact that many methodological studies have shown that Cronbach’s alpha is riddled with problems stemming from unrealistic assumptions. In many circumstances, violating these assumptions yields estimates of reliability that are too small, making measures look less reliable than they actually are. Although methodological critiques of Cronbach’s alpha are being cited with increasing frequency in empirical studies, in this tutorial we discuss how the trend is not necessarily improving methodology used in the literature. That is, many studies continue to use Cronbach’s alpha without regard for its assumptions or merely cite methodological articles advising against its use to rationalize unfavorable Cronbach’s alpha estimates. This tutorial first provides evidence that recommendations against Cronbach’s alpha have not appreciably changed how empirical studies report reliability. Then, we summarize the drawbacks of Cronbach’s alpha conceptually without relying on mathematical or simulation-based arguments so that these arguments are accessible to a broad audience. We continue by discussing several alternative measures that make less rigid assumptions which provide justifiably higher estimates of reliability compared to Cronbach’s alpha. We conclude with empirical examples to illustrate advantages of alternative measures of reliability including omega total, Revelle’s omega total, the greatest lower bound, and Coefficient H. A detailed software appendix is also provided to help researchers implement alternative methods. Translational Abstract Scales are commonly used in psychological research to measure directly unobservable constructs like motivation or depression. These scales are comprised of multiple items, each aiming to provide information about various aspects of the construct of interest. Whenever a scale is used in a psychological study, it is important to report on its reliability. Since the 1950s, the primary method for capturing reliability has been Cronbach’s alpha, a method whose status is perhaps best exemplified by its place as one of the most cited scientific articles of all-time, in any field. Despite its overwhelming popularity, the underlying assumptions of Cronbach’s alpha have been questioned recently in the statistical literature because these assumptions were commonplace 65 years ago but have largely disappeared from more modern statistical methods for constructing scales. Though the ideas in these statistical articles have the potential to significantly alter how psychological research is conducted and reported, recommendations from the statistical literature have yet to permeate the psychological literature. In this article, the goal is to demonstrate why Cronbach’s alpha is no longer the optimal method for reporting on reliability. To differentiate this article from articles appearing in the statistical literature, we approach issues with Cronbach’s alpha with very little focus on mathematical or computational detail so that the deficiencies of Cronbach’s alpha are illustrated in words and examples rather than proofs and simulations so that these ideas can impact a larger group of researchers—namely, the researchers who most often report Cronbach’s alpha.

[1]  Zinaida Benenson,et al.  Lower Bounds , 2021, Algorithms.

[2]  Ke-Hai Yuan,et al.  Robust Coefficients Alpha and Omega and Confidence Intervals With Outlying Observations and Missing Data , 2016, Educational and psychological measurement.

[3]  Richard Van Noorden,et al.  The top 100 papers , 2014, Nature.

[4]  D. Sharpe Why the resistance to statistical innovations? Bridging the communication gap. , 2013, Psychological methods.

[5]  Timothy Teo,et al.  Coefficient Alpha and Beyond: Issues and Alternatives for Educational Research , 2013 .

[6]  S. Reise,et al.  Scoring and Modeling Psychological Measures in the Presence of Multidimensionality , 2013, Journal of personality assessment.

[7]  S. Reise The Rediscovery of Bifactor Measurement Models , 2012 .

[8]  V. Savalei,et al.  When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. , 2012, Psychological methods.

[9]  Yanyan Sheng,et al.  Is Coefficient Alpha Robust to Non-Normal Data? , 2011, Front. Psychology.

[10]  Peter M. Bentler,et al.  Alpha, Dimension-Free, and Model-Based Internal Consistency Reliability , 2009, Psychometrika.

[11]  W. Revelle,et al.  Coefficients Alpha, Beta, Omega, and the glb: Comments on Sijtsma , 2009 .

[12]  Klaas Sijtsma,et al.  On the Use, the Misuse, and the Very Limited Usefulness of Cronbach’s Alpha , 2008, Psychometrika.

[13]  Ken Kelley,et al.  Methods for the Behavioral, Educational, and Social Sciences: An R package , 2007, Behavior research methods.

[14]  F. Chen Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance , 2007 .

[15]  Steven P. Reise,et al.  The role of the bifactor model in resolving dimensionality issues in health outcomes measures , 2007, Quality of Life Research.

[16]  Rik Crutzen,et al.  Time is a jailer: What do alpha and its alternatives tell us about reliability? , 2007 .

[17]  Samuel Kotz,et al.  On the η-κ distribution , 2006, IEEE Trans. Broadcast..

[18]  Hans-Georg Wolff,et al.  Exploring item and higher order factor structure with the Schmid-Leiman solution: Syntax codes for SPSS and SAS , 2005, Behavior research methods.

[19]  Jos M. F. ten Berge,et al.  The greatest lower bound to the reliability of a test and the hypothesis of unidimensionality , 2004 .

[20]  Tenko Raykov,et al.  Behavioral scale reliability and measurement invariance evaluation using latent variable modeling , 2004 .

[21]  Patrick E. Shrout,et al.  Reliability of Scales With General Structure: Point and Interval Estimation Using a Structural Equation Modeling Approach , 2002 .

[22]  G Becker,et al.  How important is transient error in estimating reliability? Going beyond simulation studies. , 2000, Psychological methods.

[23]  Alexander Shapiro,et al.  The asymptotic bias of minimum trace factor analysis, with applications to the greatest lower bound to reliability , 2000 .

[24]  Tenko Raykov,et al.  Coefficient Alpha and Composite Reliability With Interrelated Nonhomogeneous Items , 1998 .

[25]  T. Raykov Estimation of Composite Reliability for Congeneric Measures , 1997 .

[26]  N. Schmitt Uses and abuses of coefficient alpha. , 1996 .

[27]  David Thissen,et al.  Uses of Item Response Theory and the Testlet Concept in the Measurement of Psychopathology , 1996 .

[28]  Bruno D. Zumbo,et al.  Coefficient Alpha as an Estimate of Test Reliability Under Violation of Two Assumptions , 1993 .

[29]  J. M. Cortina,et al.  What Is Coefficient Alpha? An Examination of Theory and Applications , 1993 .

[30]  J. Berge,et al.  A numerical approach to the approximate and the exact minimum rank of a covariance matrix , 1991 .

[31]  L. Crocker,et al.  Introduction to Classical and Modern Test Theory , 1986 .

[32]  A. Bandura Self-efficacy: toward a unifying theory of behavioral change. , 1977, Psychological review.

[33]  William W. Rozeboom,et al.  Scaling theory and the nature of measurement , 1966, Synthese.

[34]  J. Carroll The nature of the data, or how to choose a correlation coefficient , 1961 .

[35]  John Schmid,et al.  The development of hierarchical factor solutions , 1957 .

[36]  L. Cronbach Coefficient alpha and the internal structure of tests , 1951 .

[37]  W. Revelle psych: Procedures for Personality and Psychological Research , 2017 .

[38]  W. Revelle,et al.  Individual Differences in Cognition: New Methods for Examining the Personality-Cognition Link , 2010 .

[39]  Peter M. Bentler,et al.  Covariance Structure Models for Maximal Reliability of Unit-Weighted Composites , 2007 .

[40]  Frank A. Sortino,et al.  From alpha to omega , 2001 .

[41]  W Revelle,et al.  Hierarchical Cluster Analysis And The Internal Structure Of Tests. , 1979, Multivariate behavioral research.

[42]  Uhjru 6RþDQ,et al.  Assessment of Reliability when Test Items are not Essentially τ-Equivalent , 2022 .