Heterogeneity is not always noise: lessons from improvement.

Rigorous experimental methods suppress differences among study participants (noise) to detect true intervention effects (signals). But suppressing participants' heterogeneity obscures an essential dimension of biological and clinical knowledge. Medicine is therefore ambivalent about the influence of heterogeneity on outcomes and struggles to find ways to take it properly into account in both clinical practice and research. This analysis explores the roots of that ambivalence. Drawing on the evaluation of 2 health care improvement initiatives, this article examines the unique features of improvement that help to understand heterogeneity's influence on study methods, and suggests a variety of ways to assess the effect of heterogeneity on study outcome measures.

[1]  The use of medical emergency teams in medical and surgical patients: impact of patient, nurse and organisational characteristics , 2008, Quality & Safety in Health Care.

[2]  J. Lowery,et al.  Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science , 2009, Implementation science : IS.

[3]  Shaun Treweek,et al.  What kind of randomized trials do we need? , 2009, Journal of clinical epidemiology.

[4]  Rinaldo Bellomo,et al.  The relationship between early emergency team calls and serious adverse events* , 2009, Critical care medicine.

[5]  Rebecca Horvath,et al.  Establishing a rapid response team (RRT) in an academic hospital: one year's experience. , 2006, Journal of hospital medicine.

[6]  R J Lilford,et al.  An epistemology of patient safety research: a framework for study design and interpretation. Part 2. Study design , 2008, Quality & Safety in Health Care.

[7]  A. Farmer,et al.  Designing and evaluating complex interventions to improve health care , 2007, BMJ : British Medical Journal.

[8]  P. Pronovost,et al.  Assessing safety culture: guidelines and recommendations , 2005, Quality and Safety in Health Care.

[9]  Douglas K Owens,et al.  Challenges in Systematic Reviews: Synthesis of Topics Related to the Delivery, Organization, and Financing of Health Care , 2005, Annals of Internal Medicine.

[10]  C. Meinert.  Clinical Trials Dictionary: Terminology and Usage Recommendations , 1996 .

[11]  John A Spertus,et al.  Hospital-wide code rates and mortality before and after implementation of a rapid response team. , 2008, JAMA.

[12]  Christopher H Schmid,et al.  Summing up evidence: one answer is not always enough , 1998, The Lancet.

[13]  K. Volpp,et al.  Comparative effectiveness--thinking beyond medication A versus medication B. , 2009, The New England journal of medicine.

[14]  G. Robert,et al.  Diffusion of innovations in service organizations: systematic review and recommendations. , 2004, The Milbank quarterly.

[15]  Alan Shiell,et al.  Methods for exploring implementation variation and local context within a cluster randomised community intervention trial , 2004, Journal of Epidemiology and Community Health.

[16]  Richard L Kravitz,et al.  Evidence-based medicine, heterogeneity of treatment effects, and the trouble with averages. , 2004, The Milbank quarterly.

[17]  S. Saint,et al.  How active resisters and organizational constipators affect health care-acquired infection prevention efforts. , 2009, Joint Commission journal on quality and patient safety.

[18]  Rinaldo Bellomo,et al.  The effectiveness of implementation of the medical emergency team (MET) system and factors associated with use during the MERIT study. , 2007, Critical care and resuscitation : journal of the Australasian Academy of Critical Care Medicine.

[19]  M. Finkelstein,et al.  Implementation and impact of a rapid response team in a children's hospital. , 2007, Joint Commission journal on quality and patient safety.

[20]  David B. Pillemer,et al.  Summing Up: The Science of Reviewing Research , 1984 .

[21]  F. Sebat,et al.  A multidisciplinary community hospital program for early and rapid resuscitation of shock in nontrauma patients. , 2005, Chest.

[22]  W. Shadish,et al.  Experimental and Quasi-Experimental Designs for Generalized Causal Inference , 2001 .

[23]  Simon G Thompson,et al.  Why and how Sources of Heterogeneity Should be Investigated , 2008 .

[24]  H. Heng,et al.  Heng Reductionism The Conflict Between Complex Systems and Correction , 2008 .

[25]  Trisha Greenhalgh,et al.  How do you modernize a health service? A realist evaluation of whole-scale transformation in london. , 2009, The Milbank quarterly.

[26]  Comilla Sasson,et al.  Rapid Response Teams: A Systematic Review and Meta-analysis. , 2010, Archives of internal medicine.

[27]  S. Clegg.  Making Social Science Matter: Why Social Inquiry Fails and How It Can Succeed Again , 2002 .

[28]  Jan P Vandenbroucke,et al.  Observational Research, Randomised Trials, and Two Views of Medical Science , 2008, PLoS medicine.

[29]  Keith O'Rourke,et al.  Introduction: Rationale, Potentials, and Promise of Systematic Reviews , 2008 .

[30]  K. Hillman,et al.  Findings of the First Consensus Conference on Medical Emergency Teams* , 2006, Critical care medicine.

[31]  K. Hillman,et al.  Introduction of the medical emergency team (MET) system: a cluster-randomised controlled trial , 2005, The Lancet.

[32]  M. Eccles,et al.  Planning and Studying Improvement in Patient Care: The Use of Theoretical Perspectives , 2007, The Milbank quarterly.

[33]  Elizabeth A. Hunt,et al.  Rapid response systems: A systematic review* , 2007, Critical care medicine.

[34]  R J Lilford,et al.  An epistemology of patient safety research: a framework for study design and interpretation. Part 4. One size does not fit all , 2008, Quality & Safety in Health Care.

[35]  David M Kent,et al.  Limitations of applying summary results of clinical trials to individual patients: the need for risk stratification. , 2007, JAMA.

[36]  H. Heng,et al.  Genetic and epigenetic heterogeneity in cancer: A genome‐centric perspective , 2009, Journal of cellular physiology.

[37]  K. Shojania,et al.  The tension between needing to improve care and knowing how to do it. , 2007, The New England journal of medicine.

[38]  J. Tibballs,et al.  Reduction of paediatric in-patient cardiac arrest and death with a medical emergency team: preliminary results , 2005, Archives of Disease in Childhood.

[39]  H. Montenegro,et al.  Beyond the intensive care unit: a review of interventions aimed at anticipating and preventing in-hospital cardiopulmonary arrest. , 2007, Resuscitation.

[40]  R. Brilli,et al.  Implementation of a medical emergency team in a large pediatric teaching hospital prevents respiratory and cardiopulmonary arrests outside the intensive care unit* , 2007, Pediatric critical care medicine : a journal of the Society of Critical Care Medicine and the World Federation of Pediatric Intensive and Critical Care Societies.

[41]  Paul J Sharek,et al.  Effect of a rapid response team on hospital-wide mortality and code rates outside the ICU in a Children's Hospital. , 2007, JAMA.

[42]  K. Walshe.  Understanding what works--and why--in quality improvement: the need for theory-driven evaluation. , 2007, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[43]  Daryl A Jones,et al.  Effect of the medical emergency team on long-term mortality following major surgery , 2007, Critical care.

[44]  D. Stryer,et al.  Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. , 2003, JAMA.

[45]  P. Pronovost,et al.  An intervention to decrease catheter-related bloodstream infections in the ICU. , 2006, The New England journal of medicine.

[46]  M. Pittler.  Systematic Reviews in Health Care: Meta‐analysis in Context , 2010 .

[47]  J. Swales.  The troublesome search for evidence: Three cultures in need of integration , 2000, Journal of the Royal Society of Medicine.

[48]  K. Shojania,et al.  Evidence-based quality improvement: the state of the science. , 2005, Health affairs.

[49]  Theodore Speroff,et al.  Study Designs for PDSA Quality Improvement Research , 2004, Quality management in health care.

[50]  Paul Batalden,et al.  Teaching quality improvement: the devil is in the details. , 2007, JAMA.

[51]  P. Pronovost,et al.  Translating evidence into practice: a model for large scale knowledge translation , 2008, BMJ : British Medical Journal.

[52]  Rinaldo Bellomo,et al.  Prospective controlled trial of effect of medical emergency team on postoperative morbidity and mortality rates* , 2004, Critical care medicine.

[53]  B. Thiers.  Evidence for the impact of quality improvement collaboratives: systematic review , 2009 .

[54]  Daryl A Jones,et al.  Long-term effect of a Medical Emergency Team on mortality in a teaching hospital. , 2007, Resuscitation.

[55]  Daryl A Jones,et al.  Long term effect of a medical emergency team on cardiac arrests in a teaching hospital , 2005, Critical care.

[56]  Stephen W Lagakos,et al.  Statistics in medicine--reporting of subgroup analyses in clinical trials. , 2007, The New England journal of medicine.

[57]  I. Scott,et al.  The evolving science of translating research evidence into clinical practice , 2007, ACP journal club.

[58]  Arash Rashidian,et al.  Introducing Critical Care Outreach: a ward-randomised trial of phased introduction in a general hospital , 2004, Intensive Care Medicine.

[59]  Peter J Pronovost,et al.  Reality check for checklists , 2009, The Lancet.

[60]  D. Moher,et al.  Improving the reporting of pragmatic trials: an extension of the CONSORT statement , 2008, BMJ : British Medical Journal.

[61]  Peter J Pronovost,et al.  Rapid response teams--walk, don't run. , 2006, JAMA.

[62]  Nick Tilley,et al.  Sherman vs Sherman , 2009 .

[63]  S. Harvey,et al.  Investigating the effectiveness of critical care outreach services: a systematic review , 2006, Intensive Care Medicine.

[64]  Mary Dixon-Woods,et al.  Why is Patient Safety so Hard? A Selective Review of Ethnographic Studies , 2010, Journal of health services research & policy.