The Spread of Evidence-Poor Medicine via Flawed Social-Network Analysis

The chronic widespread misuse of statistics is usually inadvertent, not intentional. We find cautionary examples in a series of recent papers by Christakis and Fowler that advance statistical arguments for the transmission via social networks of various personal characteristics, including obesity, smoking cessation, happiness, and loneliness. Those papers also assert that such influence extends to three degrees of separation in social networks. We shall show that these conclusions do not follow from Christakis and Fowler's statistical analyses. In fact, their studies even provide some evidence against the existence of such transmission. The errors that we expose arose, in part, because the assumptions behind the statistical procedures used were insufficiently examined, not only by the authors, but also by the reviewers. Our examples are instructive because the practitioners are highly reputed, their results have received enormous popular attention, and the journals that published their studies are among the most respected in the world. An educational bonus emerges from the difficulty we report in getting our critique published. We discuss the relevance of this episode to understanding statistical literacy and the role of scientific review, as well as to reforming statistics education.

[1]  J. Heckman Dummy Endogenous Variables in a Simultaneous Equation System , 1977 .

[2]  S. Glantz Biostatistics: how to detect, correct and prevent errors in the medical literature. , 1980, Circulation.

[3]  Anthony Oberschall,et al.  Notes on Social Measurement: Historical and Critical. , 1986 .

[4]  S. Zeger,et al.  Longitudinal data analysis using generalized linear models , 1986 .

[5]  H. Blalock The Real and Unrealized Contributions of Quantitative Sociology , 1989 .

[6]  Bernd Baldus Positivism's Twilight? , 1990 .

[7]  R. Smith,et al.  Where is the wisdom...? , 1991, BMJ.

[8]  L. Summers The Scientific Illusion in Empirical Macroeconomics , 1991 .

[9]  G. Taubes Epidemiology faces its limits. , 1995, Science.

[10]  D. Altman,et al.  Statistical reviewing for medical journals. , 1998, Statistics in medicine.

[11]  J. Stiglitz Lessons from the Global Financial Crisis , 2000 .

[12]  D. Altman Poor-quality medical research: what can journals do? , 2002, JAMA.

[13]  E. Tamer Incomplete Simultaneous Discrete Response Model with Multiple Equilibria , 2003 .

[14]  G. Cumming,et al.  Editors Can Lead Researchers to Confidence Intervals, but Can't Make Them Think , 2004, Psychological science.

[15]  G. Gigerenzer Mindless statistics , 2004 .

[16]  D. Freedman,et al.  Invited commentary: how far can epidemiologists get with statistical adjustment? , 2005, American journal of epidemiology.

[17]  J. Ioannidis Contradicted and initially stronger effects in highly cited clinical research. , 2005, JAMA.

[18]  J. Ioannidis Why Most Published Research Findings Are False , 2005, PLoS medicine.

[19]  David A. Freedman,et al.  Statistical Models: Theory and Practice: References , 2005 .

[20]  J. Keynes Professor Tinbergen’s Method , 2007 .

[21]  N. Christakis,et al.  The Spread of Obesity in a Large Social Network Over 32 Years , 2007, The New England journal of medicine.

[22]  D. Freedman Statistical Models and Causal Inference: On Regression Adjustments in Experiments with Several Treatments , 2008, 0803.3757.

[23]  David A. Freedman,et al.  Editorial: Oasis or Mirage? , 2008 .

[24]  N. Christakis,et al.  SUPPLEMENTARY ONLINE MATERIAL FOR: The Collective Dynamics of Smoking in a Large Social Network , 2022 .

[25]  P. Sainsbury Commentary: Understanding social network analysis , 2008, BMJ : British Medical Journal.

[26]  A. Steptoe,et al.  Happiness, social networks, and health , 2008, BMJ : British Medical Journal.

[27]  N. Christakis,et al.  Dynamic spread of happiness in a large social network: longitudinal analysis over 20 years in the Framingham Heart Study , 2008, BMJ : British Medical Journal.

[28]  Jason M. Fletcher,et al.  Detecting implausible social network effects in acne, height, and headaches: longitudinal analysis , 2008, BMJ : British Medical Journal.

[29]  N. Christakis,et al.  Estimating Peer Effects on Health in Social Networks , 2008, Journal of health economics.

[30]  Jason M. Fletcher,et al.  Is Obesity Contagious? Social Networks vs. Environmental Factors in the Obesity Epidemic , 2008, Journal of Health Economics.

[31]  K. Coombes,et al.  Deriving chemosensitivity from cell lines: Forensic bioinformatics and reproducible research in high-throughput biology , 2009, 1010.1092.

[32]  N. Christakis,et al.  Alone in the Crowd: The Structure and Spread of Loneliness in a Large Social Network , 2009 .

[33]  P. Stark,et al.  Statistical Models: Foreword to the Revised Edition , 2009 .

[34]  W. Maziak Point-counterpoint. The triumph of the null hypothesis: epidemiology in an age of change. , 2009, International journal of epidemiology.

[35]  I. N. A. C. I. J. H. Fowler Book Review: Connected: The surprising power of our social networks and how they shape our lives. , 2009 .

[36]  T. Siegfried Odds are, it's wrong: Science fails to face the shortcomings of statistics , 2010 .

[37]  J. Murabito,et al.  The Spread of Alcohol Consumption Behavior in a Large Social Network , 2010, Annals of Internal Medicine.

[38]  N. Christakis,et al.  Social network determinants of depression , 2010, Molecular Psychiatry.

[39]  D. Altman,et al.  Inadequate post-publication review of medical research , 2010, BMJ : British Medical Journal.

[40]  J. E. West,et al.  Is poor fitness contagious?: Evidence from randomly assigned friends , 2010 .

[41]  J. E. West,et al.  Is Poor Fitness Contagious? Evidence from Randomly Assigned Friends , 2010 .

[42]  J. Stiglitz Lessons from the Global Financial Crisis of 2008 , 2010 .

[43]  Susan K. Walker Connected: The Surprising Power of Our Social Networks and How They Shape Our Lives , 2011 .

[44]  Cosma Rohilla Shalizi,et al.  Homophily and Contagion Are Generically Confounded in Observational Social Network Studies , 2010, Sociological methods & research.

[45]  Marco D. Huesch,et al.  Is blood thicker than water? Peer effects in stent utilization among Floridian cardiologists. , 2011, Social science & medicine.

[46]  Brendan Nyhan,et al.  The "unfriending" problem: The consequences of homophily in friendship retention for causal estimates of social influence , 2010, Soc. Networks.

[47]  N. Christakis,et al.  Social network determinants of depression , 2011, Molecular Psychiatry.

[48]  W. Marsden I and J , 2012 .

[49]  A. Cann Replication , 2003, Principles of Molecular Virology.

[50]  Nicholas A. Christakis,et al.  Social contagion theory: examining dynamic social networks and human behavior , 2011, Statistics in medicine.

[51]  David A Freedman Oasis or Mirage ? , 2022 .