Perspective: Limiting Dependence on Nonrandomized Studies and Improving Randomized Trials in Human Nutrition Research: Why and How.

A large majority of human nutrition research uses nonrandomized observational designs, but this has led to little reliable progress. This is mostly due to many epistemologic problems, the most important of which are as follows: difficulty detecting small (or even tiny) effect sizes reliably for nutritional risk factors and nutrition-related interventions; difficulty properly accounting for massive confounding among many nutrients, clinical outcomes, and other variables; difficulty measuring diet accurately; and suboptimal research reporting. Tiny effect sizes and massive confounding are largely unfixable problems that narrowly confine the scenarios in which nonrandomized observational research is useful. Although nonrandomized studies and randomized trials have different priorities (assessment of long-term causality compared with assessment of treatment effects), the odds for obtaining reliable information with the former are limited. Randomized study designs should therefore largely replace nonrandomized studies in human nutrition research going forward. To achieve this, many of the limitations that have traditionally plagued most randomized trials in nutrition, such as small sample size, short length of follow-up, high cost, and selective reporting, among others, must be overcome. Pivotal megatrials with tens of thousands of participants and lifelong follow-up are possible in nutrition science with proper streamlining of operational costs. Fixable problems that have undermined observational research, such as dietary measurement error and selective reporting, need to be addressed in randomized trials. For focused questions in which dietary adherence is important to maximize, trials with direct observation of participants in experimental in-house settings may offer clean answers on short-term metabolic outcomes. Other study designs of randomized trials to consider in nutrition include registry-based designs and "N-of-1" designs. Mendelian randomization designs may also offer some more reliable leads for testing interventions in trials. Collectively, an improved randomized agenda may clarify many things in nutrition science that might never be answered credibly with nonrandomized observational designs.

[1]  Antoon Bronselaer,et al.  Perspective: Essential Study Quality Descriptors for Data from Nutritional Epidemiologic Research , 2017, Advances in nutrition.

[2]  J. Ioannidis,et al.  Nature, Nurture, and Cancer Risks: Genetic and Nutritional Contributions to Cancer. , 2017, Annual review of nutrition.

[3]  G. Guyatt,et al.  Perspective: Improving Nutritional Guidelines for Sustainable Health Policies: Current Status and Perspectives , 2017, Advances in nutrition.

[4]  H. Campbell,et al.  Serum uric acid levels and multiple health outcomes: umbrella review of evidence from observational studies, randomised controlled trials, and Mendelian randomisation studies , 2017, British Medical Journal.

[5]  Patrick M M Bossuyt,et al.  Waste, Leaks, and Failures in the Biomarker Pipeline. , 2017, Clinical chemistry.

[6]  L. Irwig,et al.  Beyond the treatment effect: Evaluating the effects of patient preferences in randomised trials , 2017, Statistical methods in medical research.

[7]  John P. A. Ioannidis,et al.  A manifesto for reproducible science , 2017, Nature Human Behaviour.

[8]  Eric E. Smith,et al.  Registry-based randomized controlled trials- what are the advantages, challenges, and areas for future research? , 2016, Journal of clinical epidemiology.

[9]  A. Moran,et al.  The Global Burden of Disease Study and the Preventable Burden of NCD. , 2016, Global heart.

[10]  J. Ioannidis,et al.  Randomized trials are frequently fragmented in multiple secondary publications. , 2016, Journal of clinical epidemiology.

[11]  S. Lorkowski,et al.  Perspective: NutriGrade: A Scoring System to Assess and Judge the Meta-Evidence of Randomized Controlled Trials and Cohort Studies in Nutrition Research. , 2016, Advances in nutrition.

[12]  D. Madigan,et al.  Robust empirical calibration of p‐values using observational data , 2016, Statistics in medicine.

[13]  John P. A. Ioannidis,et al.  Systematic assessment of pharmaceutical prescriptions in association with cancer risk: a method to conduct a population-wide medication-wide longitudinal study , 2016, Scientific Reports.

[14]  L. Hartling,et al.  N-of-1 trials are a tapestry of heterogeneity. , 2016, Journal of clinical epidemiology.

[15]  C. Schmid,et al.  N-of-1 trials can be aggregated to generate group mean treatment effects: a systematic review and meta-analysis. , 2016, Journal of clinical epidemiology.

[16]  E. Ravussin,et al.  Energy expenditure and body composition changes after an isocaloric ketogenic diet in overweight and obese men. , 2016, The American journal of clinical nutrition.

[17]  N. Temple How reliable are randomised controlled trials for studying the relationship between diet and disease? A narrative review , 2016, British Journal of Nutrition.

[18]  J. Ioannidis We need more randomized trials in nutrition-preferably large, long-term, and with negative results. , 2016, The American journal of clinical nutrition.

[19]  J. Ioannidis,et al.  Evolution of Reporting P Values in the Biomedical Literature, 1990-2015. , 2016, JAMA.

[20]  L. Trinquart,et al.  Why do we think we know what we know? A metaknowledge analysis of the salt controversy. , 2016, International journal of epidemiology.

[21]  Kurt Straif,et al.  Carcinogenicity of consumption of red and processed meat. , 2015, The Lancet. Oncology.

[22]  John P A Ioannidis,et al.  Assessment of vibration of effects due to model specification can demonstrate the instability of observational associations. , 2015, Journal of clinical epidemiology.

[23]  Kevin D Hall,et al.  Calorie for Calorie, Dietary Fat Restriction Results in More Body Fat Loss than Carbohydrate Restriction in People with Obesity. , 2015, Cell metabolism.

[24]  John P A Ioannidis,et al.  Evaluation of Wellness Determinants and Interventions by Citizen Scientists. , 2015, JAMA.

[25]  C. Lavie,et al.  The Inadmissibility of What We Eat in America and NHANES Dietary Data in Nutrition and Obesity Research and the Scientific Formulation of National Dietary Guidelines. , 2015, Mayo Clinic proceedings.

[26]  P. Donnan,et al.  The PRECIS-2 tool: designing trials that are fit for purpose , 2015, BMJ : British Medical Journal.

[27]  Christopher B. Granger,et al.  Registry-based randomized clinical trials—a new clinical trial paradigm , 2015, Nature Reviews Cardiology.

[28]  J. Ioannidis,et al.  Call to improve transparency of trials of non-regulated interventions , 2015, BMJ : British Medical Journal.

[29]  J. Ioannidis,et al.  Environmental risk factors and multiple sclerosis: an umbrella review of systematic reviews and meta-analyses , 2015, The Lancet Neurology.

[30]  Chirag J. Patel,et al.  Development of Exposome Correlations Globes to Map Out Environment-Wide Associations , 2014, Pacific Symposium on Biocomputing.

[31]  L. Gluud,et al.  Antioxidant supplements for prevention of mortality in healthy participants and patients with various diseases. , 2008, The Cochrane database of systematic reviews.

[32]  John P. A. Ioannidis,et al.  Big data meets public health , 2014, Science.

[33]  J. Ioannidis,et al.  Unscientific beliefs about scientific topics in nutrition. , 2014, Advances in nutrition.

[34]  D. Albanes,et al.  Effects of α‐tocopherol and β‐carotene supplementation on cancer incidence and mortality: 18‐Year postintervention follow‐up of the Alpha‐Tocopherol, Beta‐Carotene Cancer Prevention Study , 2014, International journal of cancer.

[35]  P. Miller,et al.  Low-calorie sweeteners and body weight and composition: a meta-analysis of randomized controlled trials and prospective cohort studies123 , 2014, The American journal of clinical nutrition.

[36]  John P A Ioannidis,et al.  Placing epidemiological results in the context of multiplicity and typical correlations of exposures , 2014, Journal of Epidemiology & Community Health.

[37]  John P A Ioannidis,et al.  Studying the elusive environment in large scale. , 2014, JAMA.

[38]  M. Z. Ankarfeldt Comment on "Limitations of observational evidence: implications for evidence-based dietary recommendations". , 2014, Advances in nutrition.

[39]  J. McMurray,et al.  Large streamlined trials in cardiovascular disease. , 2014, European heart journal.

[40]  Eduardo L. Franco,et al.  Making Prospective Registration of Observational Research a Reality , 2014, Science Translational Medicine.

[41]  P. Kris-Etherton,et al.  Limitations of observational evidence: implications for evidence-based dietary recommendations. , 2014, Advances in nutrition.

[42]  J. Ioannidis Implausible results in human nutrition research , 2013, BMJ.

[43]  V. Johnson Revised standards for statistical evidence , 2013, Proceedings of the National Academy of Sciences.

[44]  Ralph B D'Agostino,et al.  The randomized registry trial--the next disruptive technology in clinical research? , 2013, The New England journal of medicine.

[45]  S. Blair,et al.  Validity of U.S. Nutritional Surveillance: National Health and Nutrition Examination Survey Caloric Energy Intake Data, 1971–2010 , 2013, PloS one.

[46]  C. Gluud,et al.  Meta-Regression Analyses, Meta-Analyses, and Trial Sequential Analyses of the Effects of Supplementation with Beta-Carotene, Vitamin A, and Vitamin E Singly or in Different Combinations on All-Cause Mortality: Do We Have Evidence for Lack of Harm? , 2013, PloS one.

[47]  C. Gamble,et al.  Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias — An Updated Review , 2013, PloS one.

[48]  Brian A. Nosek,et al.  Power failure: why small sample size undermines the reliability of neuroscience , 2013, Nature Reviews Neuroscience.

[49]  Dolores Corella,et al.  Primary prevention of cardiovascular disease with a Mediterranean diet. , 2013, The New England journal of medicine.

[50]  Zhong-xiang Yuan,et al.  Effect of Antioxidant Vitamin Supplementation on Cardiovascular Outcomes: A Meta-Analysis of Randomized Controlled Trials , 2013, PloS one.

[51]  V. Prasad,et al.  Prespecified falsification end points: can they validate true observational associations? , 2013, JAMA.

[52]  John P A Ioannidis,et al.  Is everything we eat associated with cancer? A systematic cookbook review. , 2013, The American journal of clinical nutrition.

[53]  Toby J Athersuch,et al.  The role of metabolomics in characterizing the human exposome. , 2012, Bioanalysis.

[54]  P. Kris-Etherton,et al.  Exploring the Factors That Affect Blood Cholesterol and Heart Disease Risk: Is Dietary Cholesterol as Bad for You as History Leads Us to Believe?12 , 2012, Advances in nutrition.

[55]  J. Ioannidis The importance of potential studies that have not existed and registration of observational data sets. , 2012, JAMA.

[56]  I. Huybrechts,et al.  Review and evaluation of innovative technologies for measuring diet in nutritional epidemiology. , 2012, International journal of epidemiology.

[57]  S. Murphy,et al.  A "SMART" design for building individualized treatment sequences. , 2012, Annual review of clinical psychology.

[58]  Gordon H Guyatt,et al.  Credibility of claims of subgroup effects in randomised controlled trials: systematic review , 2012, BMJ : British Medical Journal.

[59]  Atul J Butte,et al.  Systematic evaluation of environmental factors: persistent pollutants and nutrients correlated with serum lipid levels , 2012, International journal of epidemiology.

[60]  Benjamin Straube,et al.  An overview of the neuro-cognitive processes involved in the encoding, consolidation, and retrieval of true and false memories , 2012, Behavioral and Brain Functions.

[61]  H. Seo,et al.  Effects of Beta-Carotene Supplements on Cancer Prevention: Meta-Analysis of Randomized Controlled Trials , 2011, Nutrition and Cancer.

[62]  J. Ioannidis,et al.  Risk factors and interventions with statistically significant tiny effects. , 2011, International journal of epidemiology.

[63]  S. Stanley Young,et al.  Deming, data and observational studies , 2011 .

[64]  R. Lau,et al.  Red and Processed Meat and Colorectal Cancer Incidence: Meta-Analysis of Prospective Studies , 2011, PloS one.

[65]  J. Ioannidis,et al.  Re: Fruit and vegetable intake and overall cancer risk in the European Prospective Investigation into Cancer and Nutrition. , 2011, Journal of the National Cancer Institute.

[66]  U. Nöthlings,et al.  Fruit and vegetable intake and overall cancer risk in the European Prospective Investigation into Cancer and Nutrition (EPIC). , 2011, Journal of the National Cancer Institute.

[67]  M. Girolami,et al.  Recommendations for Biomarker Identification and Qualification in Clinical Proteomics , 2010, Science Translational Medicine.

[68]  Kristin L. Sainani Misleading Comparisons: The Fallacy of Comparing Statistical Significance , 2010, PM & R : the journal of injury, function, and rehabilitation.

[69]  Jon Nicholl,et al.  Rethinking pragmatic randomised controlled trials: introducing the “cohort multiple randomised controlled trial” design , 2010, BMJ : British Medical Journal.

[70]  John P A Ioannidis,et al.  Researching Genetic Versus Nongenetic Determinants of Disease: A Comparison and Proposed Unification , 2009, Science Translational Medicine.

[71]  Sonia S Anand,et al.  A systematic review of the evidence supporting a causal link between dietary factors and coronary heart disease. , 2009, Archives of internal medicine.

[72]  P. Scardino,et al.  The clinically-integrated randomized trial: proposed novel method for conducting large trials at low cost , 2009, Trials.

[73]  E. Loftus,et al.  The Consequences of False Memories for Food Preferences and Choices , 2009, Perspectives on psychological science : a journal of the Association for Psychological Science.

[74]  M. Tremblay,et al.  The accuracy of self-reported smoking: a systematic review of the relationship between self-reported and cotinine-assessed smoking status. , 2009, Nicotine & tobacco research : official journal of the Society for Research on Nicotine and Tobacco.

[75]  G. Guyatt,et al.  GRADE: an emerging consensus on rating quality of evidence and strength of recommendations , 2008, BMJ : British Medical Journal.

[76]  N. Timpson,et al.  Strengthening causal inference in cardiovascular epidemiology through Mendelian randomization , 2008, Annals of medicine.

[77]  J. Ioannidis,et al.  Nested Randomized Trials in Large Cohorts and Biobanks: Studying the Health Effects of Lifestyle Factors , 2008, Epidemiology.

[78]  J. Ioannidis,et al.  Persistence of contradicted claims in the literature. , 2007, JAMA.

[79]  Jonathan A C Sterne,et al.  The impact of residual and unmeasured confounding in epidemiologic studies: a simulation study. , 2007, American journal of epidemiology.

[80]  Thomas A Trikalinos,et al.  Selective discussion and transparency in microarray research findings for cancer outcomes. , 2007, European journal of cancer.

[81]  A. Kiss,et al.  Eligibility criteria of randomized controlled trials published in high-impact general medical journals: a systematic sampling review. , 2007, JAMA.

[82]  Marcia K. Johnson,et al.  Memory and Reality , 2022 .

[83]  J. Gustafsson,et al.  The Women's Health Initiative. What is on trial: nutrition and chronic disease? Or misinterpreted science, media havoc and the sound of silence from peers? , 2006, Public Health Nutrition.

[84]  J. Habbema,et al.  Subgroup analyses in therapeutic cardiovascular clinical trials: are most of them misleading? , 2006, American heart journal.

[85]  J. Ioannidis Contradicted and initially stronger effects in highly cited clinical research. , 2005, JAMA.

[86]  John D Potter,et al.  Influence of demographic, physiologic, and psychosocial variables on adherence to a yearlong moderate-intensity exercise trial in postmenopausal women. , 2004, Preventive medicine.

[87]  Harlan M Krumholz,et al.  Participation in cancer clinical trials: race-, sex-, and age-based disparities. , 2004, JAMA.

[88]  K. Courneya,et al.  Three independent factors predicted adherence in a randomized controlled trial of resistance exercise training among prostate cancer survivors. , 2004, Journal of clinical epidemiology.

[89]  S. Ebrahim,et al.  Mendelian randomization: prospects, potentials, and limitations. , 2004, International journal of epidemiology.

[90]  Dietary adherence in the Women's Health Initiative Dietary Modification Trial. , 2004, Journal of the American Dietetic Association.

[91]  Nea Malila,et al.  Incidence of cancer and mortality following alpha-tocopherol and beta-carotene supplementation: a postintervention follow-up. , 2003, JAMA.

[92]  L. Kohlmeier,et al.  Energy adjustment does not control for differential recall bias in nutritional epidemiology. , 1998, Journal of clinical epidemiology.

[93]  J. Lau,et al.  The impact of high-risk patients on the results of clinical trials. , 1997, Journal of clinical epidemiology.

[94]  D. DeMets,et al.  Surrogate End Points in Clinical Trials: Are We Being Misled? , 1996, Annals of Internal Medicine.

[95]  D. Albanes,et al.  The effect of vitamin E and beta carotene on the incidence of lung cancer and other cancers in male smokers. , 1994, The New England journal of medicine.

[96]  A. Kriska,et al.  A randomized exercise trial in older women: increased activity over two years and the factors associated with compliance. , 1986, Medicine and science in sports and exercise.

[97]  Robert Rosenthal,et al.  The Volunteer Subject , 1965 .