Qualitative Comparative Analysis in Critical Perspective

Qualitative comparative analysis (QCA) appears to offer a systematic means for case-oriented analysis. The method not only offers to provide a standardized procedure for qualitative research but also serves, to some, as an instantiation of deterministic methods. Others, however, contest QCA because of its deterministic lineage. Multiple other issues surrounding QCA, such as its response to measurement error and its ability to ascertain asymmetric causality, are also matters of interest. Existing research has demonstrated the use of QCA on real data, but such data do not allow one to establish the method’s efficacy, because the true causes of real social phenomena are always contestable. In response, the authors analyze several simulated data sets for which true causal processes are known. They find that QCA finds the correct causal story only 3 times across 70 different solutions, and even these rare successes, on closer examination, actually reveal additional fundamental problems with the method. Further epistemological analyses of the results find key problems with QCA’s stated epistemology, and results indicate that QCA fails even when its stated epistemological claims are ontologically accurate. Thus, the authors conclude that analysts should reject both QCA and its epistemological justifications in favor of existing effective methods and epistemologies for qualitative research.

[1]  S. Lieberson,et al.  Making It Count: The Improvement of Social Research and Theory. , 1987 .

[2]  W. P. Rogers,et al.  Report of the Presidential Commission on the Space Shuttle Challenger Accident, Volume 1 , 1986 .

[3]  Lee Rainwater,et al.  What Is a Good Job? A New Measure of Labor-Market Success , 1988, American Journal of Sociology.

[4]  Samuel R. Lucas An inconvenient dataset: bias and inappropriate inference with the multilevel model , 2013, Quality & Quantity.

[5]  Liz Stanley,et al.  Big structures, large processes, huge comparisons , 2009 .

[6]  I. Kawachi,et al.  Race/Ethnic and Nativity Disparities in Child Overweight in the United States and England , 2012, The Annals of the American Academy of Political and Social Science.

[7]  Carsten Q. Schneider,et al.  Qualitative Comparative Analysis (QCA) and Fuzzy-Sets: Agenda for a Research Approach and a Data Analysis Technique , 2010 .

[8]  P. Pronovost,et al.  Critical pathway effectiveness: assessing the impact of patient, hospital care, and pathway characteristics using qualitative comparative analysis. , 2005, Health services research.

[9]  R. Hodson Some considerations concerning the functional form of earnings , 1985 .

[10]  Stanley Lieberson,et al.  Small N's and Big Conclusions: An Examination of the Reasoning in Comparative Studies Based on a Small Number of Cases , 1991 .

[11]  M. Sobel,et al.  Identification Problems in the Social Sciences , 1996 .

[12]  Edward A. Suchman,et al.  Studies in Social Psychology in World War II. Vol. I: The American Soldier: Adjustment during Army Life , 1951 .

[13]  Samuel R. Lucas Beyond the existence proof: ontological conditions, epistemological implications, and in-depth interview research , 2014 .

[14]  J. R. Warren,et al.  4. Socioeconomic Indexes for Occupations: A Review, Update, and Critique , 1997 .

[15]  Charles C. Ragin,et al.  The Limitations of Net-Effects Thinking , 2006 .

[16]  L. Tolstoy,et al.  War and peace : the Maude translation, backgrounds and sources, criticism , 1996 .

[17]  R. Gott,et al.  The Roles of Substantive and Procedural Understanding in Open-Ended Science Investigations: Using Fuzzy Set Qualitative Comparative Analysis to Compare Two Different Tasks , 2009 .

[18]  J. Mahoney Strategies of Causal Inference in Small-N Analysis , 2000 .

[19]  M. Mann The Dark Side of Democracy: Explaining Ethnic Cleansing , 2004 .

[20]  Margaret R. Somers Genealogies of Citizenship: Markets, Statelessness, and the Right to Have Rights , 2008 .

[21]  W. Cleveland Robust Locally Weighted Regression and Smoothing Scatterplots , 1979 .

[22]  Developed and Democratic Countries' Policy-making on Dispute Settlement in the GATT/WTO: , 2009 .

[23]  Kyle C. Longest,et al.  Fuzzy: A Program for Performing Qualitative Comparative Analyses (QCA) in Stata , 2008, The Stata Journal: Promoting communications on statistics and Stata.

[24]  Constance F. Citro,et al.  Measuring poverty : a new approach , 1995 .

[25]  A. J. Grimes Normal Accidents: Living with High Risk Technologies , 1985 .

[26]  Stephen Vaisey Structure, Culture, and Community: The Search for Belonging in 50 Urban Communes , 2007 .

[27]  Benoît Rihoux,et al.  Innovative Comparative Methods for policy Analysis. Beyond the Quantitative-Qualitative Divide , 2006 .

[28]  Shu-Ling Tsai,et al.  A Model of Stratification with Response Error in Social and Psychological Variables , 1983 .

[29]  Robert M. Hauser,et al.  Response Errors of Black and Nonblack Males in Models of the Intergenerational Transmission of Socioeconomic Status , 1977, American Journal of Sociology.

[30]  R. Hauser On "Stratification in a Dual Economy" , 1980 .

[31]  Charles C. Ragin Core versus tangential assumptions in comparative research , 2005 .

[32]  A. J. Walker,et al.  The Dyad as the Unit of Analysis: Conceptual and Methodological Issues , 1982 .

[33]  Edward R. Tufte Visual explanations: images and quantities, evidence and narrative , 1997 .

[34]  Lawrence L. Wu Event History Models for Life Course Analysis , 2003 .

[35]  D. Pregibon Goodness of Link Tests for Generalized Linear Models , 1980 .

[36]  Geert Van Hootegem,et al.  Comparative configurational case analysis of ergonomic injuries , 2007 .

[37]  Don E. Lindley Becker, Howard S. , 2014, SAGE Research Methods Foundations.

[38]  Christopher Winship,et al.  Counterfactuals and Causal Inference: Methods and Principles for Social Research , 2007 .

[39]  D. Firth Bias reduction of maximum likelihood estimates , 1993 .

[40]  Jerald Hage,et al.  The Effect of Human Capital and State Intervention on the Performance of Medical Systems , 1996 .

[41]  S. Morris The road to hell? , 2006, Nature.

[42]  Yu-wei Xie,et al.  THE LOG-MULTIPLICATIVE LAYER EFFECT MODEL FOR COMPARING MOBILITY TABLES* , 1992 .

[43]  A. George,et al.  Case Studies and Theory Development in the Social Sciences , 2005 .

[44]  C. Glymour,et al.  STATISTICS AND CAUSAL INFERENCE , 1985 .

[45]  M. Yenidunya,et al.  Is it a matter of habit? , 2015, Brazilian Journal of Anesthesiology.

[46]  R. Herrnstein,et al.  The bell curve : intelligence and class structure in American life , 1995 .

[47]  G. Gigerenzer From Tools to Theories: A Heuristic of Discovery in Cognitive Psychology. , 1991 .

[48]  Benoît Rihoux,et al.  Qualitative Comparative Analysis (QCA) : State of the Art and Prospects , 2004 .

[49]  O. D. Duncan How Destination Depends on Origin in the Occupational Mobility Table , 1979, American Journal of Sociology.

[50]  Sidney Pollard,et al.  States and Social Revolutions. A Comparative Analysis of France, Russia, and China. , 1980 .

[51]  W. Sewell :The Familial State: Ruling Families and Merchant Capitalism in Early Modern Europe , 2009 .

[52]  A. Abbott Transcending General Linear Reality , 1988 .

[53]  Jason Seawright,et al.  Qualitative comparative analysis vis-à-vis regression , 2005 .

[54]  Stanley Lieberson,et al.  More on the Uneasy Case for Using Mill-Type Methods in Small-N Comparative Studies , 1994 .

[55]  J. M. D. Meiklejohn,et al.  The Critique of Pure Reason , 2020, A Commentary on Kant’s Critique of Judgment.

[56]  P. Lazarsfeld The American Soldier An Expository Review , 2016 .

[57]  J. Pearl 3. The Foundations of Causal Inference , 2010 .

[58]  G. Steinmetz The Devil's Handwriting: Precoloniality and the German Colonial State in Qingdao, Samoa, and Southwest Africa , 2007 .

[59]  J. Savolainen The Rationality of Drawing Big Conclusions Based on Small Samples: In Defense of Mill's Methods , 1994 .

[60]  Bernard Grofman,et al.  An Introduction to Crisp Set QCA, with a Comparison to Binary Logistic Regression , 2009 .

[61]  Wade Robison Representation and misrepresentation: Tufte and the Morton Thiokol engineers on the Challenger , 2002, Science and engineering ethics.

[62]  H. Reichenbach Experience and Prediction. An Analysis of the Foundations and the Structure of Knowledge , 1938 .

[63]  Frederick Mosteller,et al.  Representative Sampling, IV: The History of the Concept in Statistics, 1895-1939 , 1980 .

[64]  J. Mill A system of logic, ratiocinative and inductive : being a connected view of the principles of evidence, and the methods of scientific investigation. , 2011 .

[65]  M. Schemper,et al.  A solution to the problem of separation in logistic regression , 2002, Statistics in medicine.

[66]  Sandra Lowe,et al.  Redesigning Social Inquiry Fuzzy Sets And Beyond , 2016 .

[67]  E. Suchman,et al.  The American soldier: Adjustment during army life. (Studies in social psychology in World War II), Vol. 1 , 1949 .

[68]  Carsten Q. Schneider,et al.  Standards of Good Practice in Qualitative Comparative Analysis (QCA) and Fuzzy-Sets , 2010 .

[69]  David W. Hosmer,et al.  Applied Logistic Regression , 1991 .

[70]  John Gribbin,et al.  In search of Schrödinger's cat : quantum physics and reality , 1984 .

[71]  HighWire Press Philosophical Transactions of the Royal Society of London , 1781, The London Medical Journal.

[72]  M. Stenbeck Duncan, Otis Dudley , 2020, SAGE Research Methods Foundations.

[73]  Jerome Karabel,et al.  The Chosen: The Hidden History of Admission and Exclusion at Harvard, Yale, and Princeton , 2005 .

[74]  Clifford C. Clogg,et al.  Some Models for the Analysis of Association in Multiway Cross-Classifications Having Ordered Categories , 1982 .

[75]  Gary Goertz,et al.  A Tale of Two Cultures: Contrasting Quantitative and Qualitative Research , 2006, Political Analysis.

[76]  J. Mahoney Toward a Unified Theory of Causality , 2008 .

[77]  S. R. Searle,et al.  Applied Linear Regression Models, 2nd edition , 2016 .

[78]  E. Amenta,et al.  Wage Wars: Institutional Politics, WPA Wages, and the Struggle for U.S. Social Policy , 2000, American Sociological Review.

[79]  Adrian Duşa,et al.  A mathematical approach to the boolean minimization problem , 2008 .

[80]  F. Dyson,et al.  A Determination of the Deflection of Light by the Sun's Gravitational Field, from Observations Made at the Total Eclipse of May 29, 1919 , 1920 .

[81]  B. Gran,et al.  The Office of the Children’s Ombudsperson: Children’s Rights and Social-Policy Innovation , 2003 .

[82]  H. Ganzeboom COMPARATIVE INTERGENERATIONA L STRATIFICATION RESEARCH: Three Generations and Beyond , 1991 .

[83]  Benoît Rihoux,et al.  Innovative Comparative Methods for Policy Analysis , 2006 .

[84]  A. Abbott,et al.  Measuring Resemblance in Sequence Data: An Optimal Matching Analysis of Musicians' Careers , 1990, American Journal of Sociology.

[85]  Michael P. Maren,et al.  The road to hell .... , 1992 .

[86]  Robert M. Hauser,et al.  Post-High School Plans and Aspirations of Black and White High School Seniors: 1976-86 , 1991 .

[87]  P. Lazarsfeld The American Solidier—An Expository Review , 1949 .

[88]  G. John CURRENT ISSUES IN COMPARATIVE MACROSOCIOLOGY , 1997 .

[89]  E. M. Beck,et al.  Stratification in a Dual Economy: A Sectoral Model of Earnings Determination. , 1978 .

[90]  Lasse Cronqvist,et al.  Determining the Conditions of HIV/AIDS Prevalence in Sub-Saharan Africa , 2006 .

[91]  J. Mahoney Nominal, Ordinal, and Narrative Appraisal in Macrocausal Analysis , 1999, American Journal of Sociology.

[92]  Benoît Rihoux,et al.  Qualitative Comparative Analysis (QCA) As an Approach , 2009 .

[93]  Charles C. Ragin,et al.  Qualitative Comparative Analysis Using Fuzzy Sets (fsQCA) , 2008 .

[94]  William Roberts Clark,et al.  A Simple Multivariate Test for Asymmetric Hypotheses , 2006, Political Analysis.

[95]  Christopher H. Achen Two cheers for Charles Ragin , 2005 .

[96]  A. Goldberger A course in econometrics , 1991 .

[97]  Otis Dudley Duncan,et al.  A socioeconomic index for all occupations. , 1996 .

[98]  Diane Vaughan,et al.  The Challenger Launch Decision: Risky Technology, Culture, and Deviance at NASA , 1996 .

[99]  David Hachen,et al.  The American Class Structure , 1957 .