Evidence-Based Medicine Evidence-Based Medicine , Systematic Reviews , and Guidelines in Interventional Pain Management : Part 6 . Systematic Reviews and Meta-Analyses of Observational Studies

Observational studies provide an important source of information when randomized controlled trials (RCTs) cannot or should not be undertaken, provided that the data are analyzed and interpreted with special attention to bias. Evidence-based medicine (EBM) stresses the examination of evidence from clinical research and describes it as a shift in medical paradigm, in contrast to intuition, unsystematic clinical experience, and pathophysiologic rationale. While the importance of randomized trials has been created by the concept of the hierarchy of evidence in guiding therapy, much of the medical research is observational. The reporting of observational research is often not detailed and clear enough with insufficient quality and poor reporting, which hampers the assessment of strengths and weaknesses of the study and the generalizability of the mixed results. Thus, in recent years, progress and innovations in health care are measured by systematic reviews and meta-analyses. A systematic review is defined as, "the application of scientific strategies that limit bias by the systematic assembly, clinical appraisal, and synthesis of all relevant studies on a specific topic." Meta-analysis usually is the final step in a systematic review. Systematic reviews and meta-analyses are labor intensive, requiring expertise in both the subject matter and review methodology, and also must follow the rules of EBM which suggests that a formal set of rules must complement medical training and common sense for clinicians to integrate the results of clinical research effectively. While expertise in the review methods is important, the expertise in the subject matter and technical components is also crucial. Even though, systematic reviews and meta-analyses, specifically of RCTs, have exploded, the quality of the systematic reviews is highly variable and consequently, the opinions reached of the same studies are quite divergent. Numerous deficiencies have been described in methodologic assessment of the quality of the individual articles. Consequently, observational studies can provide an important complementary source of information, provided that the data are analyzed and interpreted in the context of confounding bias to which they are prone. Appropriate systematic reviews of observational studies, in conjunction with RCTs, may provide the basis for elimination of a dangerous discrepancy between the experts and the evidence. Steps in conducting systematic reviews of observational studies include planning, conducting, reporting, and disseminating the results. MOOSE, or Meta-analysis of Observational Studies in Epidemiology, a proposal for reporting contains specifications including background, search strategy, methods, results, discussion, and conclusion. Use of the MOOSE checklist should improve the usefulness of meta-analysis for authors, reviewers, editors, readers, and decision-makers. This manuscript describes systematic reviews and meta-analyses of observational studies. Authors frequently utilize RCTs and observational studies in one systematic review; thus, they should also follow the reporting standards of the Quality of Reporting of Meta-analysis (QUOROM) statement, which also provides a checklist. A combined approach of QUOROM and MOOSE will improve reporting of systematic reviews and lead to progress and innovations in health care.

[1]  M. Kendall,et al.  The Logic of Scientific Discovery. , 1959 .

[2]  A. B. Hill.  The Environment and Disease: Association or Causation? , 1965, Proceedings of the Royal Society of Medicine.

[3]  C. Hempel.  Philosophy of Natural Science , 1966 .

[4]  Karl R. Popper.  The Logic of Scientific Discovery. , 1977 .

[5]  T. Takaro,et al.  Treatment of chronic stable angina. A preliminary report of survival data of the randomized Veterans Administration cooperative study. , 1977, The New England journal of medicine.

[6]  T C Chalmers,et al.  Evidence favoring the use of anticoagulants in the hospital phase of acute myocardial infarction. , 1977, The New England journal of medicine.

[7]  Gregg B. Jackson,et al.  Methods for Integrative Reviews , 1980 .

[8]  J. Lind.  A Treatise on the scurvy : in three parts : containing an inquiry into the nature, causes, and cures of that disease : together with a critical and chronological view of what has been published on the subject , 1980 .

[9]  T C Chalmers,et al.  Randomized versus historical controls for clinical trials. , 1982, The American journal of medicine.

[10]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .

[11]  G. Torrance.  Measurement of health state utilities for economic appraisal. , 1986, Journal of health economics.

[12]  T. Chalmers,et al.  Meta-analyses of randomized controlled trials. , 1987, The New England journal of medicine.

[13]  D. Revicki,et al.  The Method of Information Synthesis and Its Use in the Assessment of Health Care Technology , 1987, International Journal of Technology Assessment in Health Care.

[14]  C. Mulrow.  The medical review article: state of the science. , 1987, Annals of internal medicine.

[15]  J. Parvizi,et al.  Total Hip Arthroplasty , 1988 .

[16]  Harris Cooper,et al.  Influences on the Outcome of Literature Searches for Integrative Research Reviews , 1989 .

[17]  I. Chalmers,et al.  Retrospective and prospective identification of unpublished controlled trials: lessons from a survey of obstetricians and pediatricians. , 1989, Pediatrics.

[18]  D B Rubin,et al.  Multiple imputation in health-care databases: an overview and some applications. , 1991, Statistics in medicine.

[19]  Marie E. Latulippe,et al.  A controlled trial of corticosteroid injections into facet joints for chronic low back pain. , 1991, The New England journal of medicine.

[20]  A. Baron,et al.  A Method to Describe Physician Decision Thresholds and Its Application in Examining the Diagnosis of Coronary Artery Disease Based on Exercise Treadmill Testing , 1992, Medical decision making : an international journal of the Society for Medical Decision Making.

[21]  A. Detsky,et al.  Evidence-based medicine. A new approach to teaching the practice of medicine. , 1992, JAMA.

[22]  G. Guyatt,et al.  The Science of Reviewing Research a , 1993, Annals of the New York Academy of Sciences.

[23]  Emile Letournel,et al.  Fractures of the Acetabulum , 1981, Springer Berlin Heidelberg.

[24]  Crossover Comparison of Drug Information Online Database Vendors: Dialog and Medlars , 1993, The Annals of pharmacotherapy.

[25]  R. Kaplan.  Modern methods of searching the medical literature. , 1993, The Medical journal of Australia.

[26]  A D Oxman,et al.  Systematic Reviews: Checklists for review articles , 1994, BMJ.

[27]  T R Einarson,et al.  Quality of nonstructured and structured abstracts of original research articles in the British Medical Journal, the Canadian Medical Association Journal and the Journal of the American Medical Association. , 1994, CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne.

[28]  K. Dickersin,et al.  Systematic Reviews: Identifying relevant studies for systematic reviews , 1994 .

[29]  H J Eysenck,et al.  Systematic Reviews: Meta-analysis and its problems , 1994, BMJ.

[30]  C. Mann.  Can meta-analysis make policy? , 1994, Science.

[31]  Performance of online biomedical databases in rheumatology. , 1994, The Journal of rheumatology.

[32]  Recommendations guiding physicians in biomedical research involving human subjects. World Medical Association Declaration of Helsinki. , 1994, Le Journal medical libanais. The Lebanese medical journal.

[33]  A R Feinstein,et al.  Use of methodological standards in diagnostic test research. Getting better but still not good. , 1995, JAMA.

[34]  D. Cook,et al.  Methodologic guidelines for systematic reviews of randomized control trials in health care from the Potsdam Consultation on Meta-Analysis. , 1995, Journal of clinical epidemiology.

[35]  R. J. Hayes,et al.  Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. , 1995, JAMA.

[36]  W. Richardson,et al.  The well-built clinical question: a key to evidence-based decisions. , 1995, ACP journal club.

[37]  R. Horton.  The rhetoric of research , 1995, BMJ.

[38]  M. Clarke,et al.  Identifying relevant studies for systematic reviews , 1995, BMJ.

[39]  R. Slavin.  Best evidence synthesis: an intelligent alternative to meta-analysis. , 1995, Journal of clinical epidemiology.

[40]  H S Sacks,et al.  Meta-analysis: an update. , 1996, The Mount Sinai journal of medicine, New York.

[41]  J Savulescu,et al.  Are research ethics committees behaving unethically? Some suggestions for improving performance and accountability , 1996, BMJ.

[42]  S. Walter,et al.  Empirical evidence of bias in infertility research: overestimation of treatment effect in crossover trials using pregnancy as the outcome measure. , 1996, Fertility and sterility.

[43]  N. Black.  Why we need observational studies to evaluate the effectiveness of health care , 1996, BMJ.

[44]  James Hartley,et al.  Obtaining information accurately and quickly: are structured abstracts more efficient? , 1996, J. Inf. Sci..

[45]  J. Matta.  Fractures of the Acetabulum: Accuracy of Reduction and Clinical Results in Patients Managed Operatively within Three Weeks after the Injury* , 1996, The Journal of bone and joint surgery. American volume.

[46]  M. Gardner,et al.  More informative abstracts revisited. , 1990, The Cleft palate-craniofacial journal : official publication of the American Cleft Palate-Craniofacial Association.

[47]  D. Moher,et al.  Completeness of reporting of trials published in languages other than English: implications for conduct and reporting of systematic reviews , 1996, The Lancet.

[48]  G Lewis,et al.  Putting trials on trial--the costs and consequences of small trials in depression: a systematic review of methodology. , 1997, Journal of epidemiology and community health.

[49]  Peter Langhorne,et al.  Integrating Heterogeneous Pieces of Evidence in Systematic Reviews , 1997, Annals of Internal Medicine.

[50]  S. Carette,et al.  Epidural corticosteroid injections for sciatica due to herniated nucleus pulposus. , 1997, The New England journal of medicine.

[51]  C. Counsell,et al.  Formulating Questions and Locating Primary Studies for Inclusion in Systematic Reviews , 1997, Annals of Internal Medicine.

[52]  F. Davidoff,et al.  Changes to manuscripts during the editorial process: characterizing the evolution of a clinical paper. , 1998, JAMA.

[53]  Susan R. Wilson,et al.  Review of the usefulness of contacting other experts when conducting a literature search for systematic reviews , 1998, BMJ.

[54]  A D Oxman,et al.  The unpredictability paradox: review of empirical comparisons of randomised and non-randomised clinical trials , 1998, BMJ.

[55]  D. Moher,et al.  Guides for reading and interpreting systematic reviews: I. Getting started. , 1998, Archives of pediatrics & adolescent medicine.

[56]  D. Cook,et al.  Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? , 1998, The Lancet.

[57]  K N Lohr,et al.  Health policy issues and applications for evidence-based medicine and clinical practice guidelines. , 1998, Health policy.

[58]  C. Adams,et al.  Content and quality of 2000 controlled trials in schizophrenia over 50 years , 1998, BMJ.

[59]  A R Jadad,et al.  Guides for reading and interpreting systematic reviews: II. How did the authors find the studies and assess their quality? , 1998, Archives of pediatrics & adolescent medicine.

[60]  A R Jadad,et al.  Methodology and reports of systematic reviews and meta-analyses: a comparison of Cochrane reviews with articles published in paper-based journals. , 1998, JAMA.

[61]  D. Moher,et al.  Guides for reading and interpreting systematic reviews: III. How did the authors synthesize the data and make their conclusions? , 1998, Archives of pediatrics & adolescent medicine.

[62]  P. Peduzzi,et al.  Twenty-two-year follow-up in the VA Cooperative Study of Coronary Artery Bypass Surgery for Stable Angina. , 1998, The American journal of cardiology.

[63]  Deborah J. Cook,et al.  Systematic Reviews: Synthesis of Best Evidence for Health Care Decisions , 1998, Annals of Internal Medicine.

[64]  T. Kiviluoto,et al.  Randomised trial of laparoscopic versus open cholecystectomy for acute and gangrenous cholecystitis , 1998, The Lancet.

[65]  E. Carragee,et al.  False-positive findings on lumbar discography. Reliability of subjective concordance assessment during provocative disc injection. , 1999, Spine.

[66]  M Lipsett,et al.  Occupational exposure to diesel exhaust and lung cancer: a meta-analysis. , 1999, American journal of public health.

[67]  I. Olkin,et al.  Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement , 1999, The Lancet.

[68]  David Moher,et al.  The Medical Review Article Revisited: Has the Science Improved? , 1999, Annals of Internal Medicine.

[69]  I. Shrier,et al.  Stretching before exercise does not reduce the risk of local muscle injury: a critical review of the clinical and basic science literature. , 1999, Clinical journal of sport medicine : official journal of the Canadian Academy of Sport Medicine.

[70]  P. Bossuyt,et al.  Empirical evidence of design-related bias in studies of diagnostic tests. , 1999, JAMA.

[71]  M. Egger,et al.  The hazards of scoring the quality of clinical trials for meta-analysis. , 1999, JAMA.

[72]  F. Song,et al.  QUALITY-ASSESSED REVIEWS OF HEALTH CARE INTERVENTIONS AND THE DATABASE OF ABSTRACTS OF REVIEWS OF EFFECTIVENESS (DARE) , 1999, International Journal of Technology Assessment in Health Care.

[73]  Warren S. Browner,et al.  Publishing and presenting clinical research , 1999 .

[74]  M Docherty,et al.  The case for structuring the discussion of scientific papers , 1999, BMJ.

[75]  A R Feinstein,et al.  Clinical epidemiological quality in molecular genetic research: the need for methodological standards. , 1999, JAMA.

[76]  M. Hotopf,et al.  Pragmatic randomised controlled trials in psychiatry , 1999, British Journal of Psychiatry.

[77]  C. Adams,et al.  Searching the right database. A comparison of four databases for psychiatry journals. , 1999, Health libraries review.

[78]  M. Liebergall,et al.  Acetabular fractures. Clinical outcome of surgical treatment. , 1999, Clinical orthopaedics and related research.

[79]  R D Herbert,et al.  A randomized trial of preexercise stretching for prevention of lower-limb injury. , 2000, Medicine and science in sports and exercise.

[80]  I. Olkin,et al.  Meta-analysis of observational studies in epidemiology - A proposal for reporting , 2000 .

[81]  B. Charlton,et al.  New perspectives in the evidence-based healthcare debate. , 2000, Journal of evaluation in clinical practice.

[82]  A R Jadad,et al.  Systematic reviews and meta-analyses on treatment of asthma: critical evaluation , 2000, BMJ : British Medical Journal.

[83]  T A Sheldon,et al.  A systematic review of comparisons of effect sizes derived from randomised and non-randomised studies. , 2000, Health technology assessment.

[84]  J. Watson,et al.  Open Reduction and Internal Fixation of Posterior Wall Fractures of the Acetabulum , 2000, Clinical orthopaedics and related research.

[85]  T. Greene.  Are Observational Studies ‘Just as Effective’ as Randomized Clinical Trials? , 2000, Blood Purification.

[86]  J. Concato,et al.  Randomized, controlled trials, observational studies, and the hierarchy of research designs. , 2000, The New England journal of medicine.

[87]  R. Kunz,et al.  Observational studies and randomized trials. , 2000, The New England journal of medicine.

[88]  Douglas G Altman,et al.  Systematic reviews in health care: Assessing the quality of controlled clinical trials. , 2001, BMJ.

[89]  L. Manchikanti,et al.  Cochrane Review by Nelemans et al. , 2001, Spine.

[90]  J. Ioannidis,et al.  Comparison of evidence of treatment effects in randomized and nonrandomized studies. , 2001, JAMA.

[91]  E. Funai,et al.  Distribution of Study Designs in Four Major US Journals of Obstetrics and Gynecology , 2001, Gynecologic and Obstetric Investigation.

[92]  A. Berg,et al.  Introducing the third U.S. preventive services task force , 2001 .

[93]  Douglas G. Altman,et al.  Systematic Reviews in Health Care: Meta-Analysis in Context: Second Edition , 2008 .

[94]  Graham A. Colditz,et al.  Systematic Reviews in Health Care: Frontmatter , 2001 .

[95]  P Middleton,et al.  Quality of Cochrane reviews: assessment of sample from 1998 , 2001, BMJ : British Medical Journal.

[96]  D Moher,et al.  The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. , 2001, Annals of internal medicine.

[97]  F. Sturmans,et al.  Injection therapy for subacute and chronic benign low back pain. , 2001, Spine.

[98]  R. Herbert,et al.  Effects of stretching before and after exercising on muscle soreness and risk of injury: systematic review , 2002, BMJ : British Medical Journal.

[99]  G. Antes,et al.  Nearly half of dental randomized controlled trials published in German are not included in Medline. , 2002, European journal of oral sciences.

[100]  C Michael Stein,et al.  COX-2 selective non-steroidal anti-inflammatory drugs and risk of serious coronary heart disease , 2002, The Lancet.

[101]  Peter Jüni,et al.  Direction and impact of language bias in meta-analyses of controlled trials: empirical study. , 2002, International journal of epidemiology.

[102]  Jane Garner,et al.  The case against ‘the evidence’: A different perspective on evidence-based medicine , 2002, British Journal of Psychiatry.

[103]  P. Tugwell,et al.  A Comparison of the Quality of Cochrane Reviews and Systematic Reviews Published in Paper-Based Journals , 2002, Evaluation & the health professions.

[104]  G. O’Keefe,et al.  Methodologic quality and genotyping reproducibility in studies of tumor necrosis factor -308 G-->A single nucleotide polymorphism and bacterial sepsis: implications for studies of complex traits. , 2003, Critical care medicine.

[105]  P. Royle,et al.  Literature searching for clinical and cost-effectiveness studies used in health technology assessment reports carried out for the National Institute for Clinical Excellence appraisal system. , 2003, Health technology assessment.

[106]  T. Diamond,et al.  Management of acute osteoporotic vertebral fractures: a nonrandomized trial comparing percutaneous vertebroplasty with conservative therapy. , 2003, The American journal of medicine.

[107]  David Moher,et al.  Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Standards for Reporting of Diagnostic Accuracy. , 2003, Clinical chemistry.

[108]  T P Klassen,et al.  The inclusion of reports of randomised trials published in languages other than English in systematic reviews. , 2003, Health technology assessment.

[109]  J. Bulman,et al.  Further statistics in dentistry Part 8: Systematic reviews and meta-analyses , 2003, British Dental Journal.

[110]  P. Bossuyt,et al.  BMC Medical Research Methodology , 2002 .

[111]  J. L. Marsh,et al.  Methodologic Issues in Observational Studies , 2003, Clinical orthopaedics and related research.

[112]  Cochrane Reviews: new blocks on the kids , 2003, British journal of sports medicine.

[113]  J. Sterne,et al.  How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. , 2003, Health technology assessment.

[114]  E. Topol,et al.  Use of antioxidant vitamins for the prevention of cardiovascular disease: meta-analysis of randomised trials , 2003, The Lancet.

[115]  Julie Gilchrist,et al.  The impact of stretching on sports injury risk: a systematic review of the literature. , 2004, Medicine and science in sports and exercise.

[116]  T. Einarson,et al.  Quality assessment of meta-analyses of RCTs of pharmacotherapy in major depressive disorder , 2004, Current medical research and opinion.

[117]  K. Lohr.  Rating the strength of scientific evidence: relevance for quality improvement programs. , 2004, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[118]  Paul Glasziou,et al.  Assessing the quality of research , 2004, BMJ : British Medical Journal.

[119]  P. Bossuyt,et al.  Sources of Variation and Bias in Studies of Diagnostic Accuracy , 2004, Annals of Internal Medicine.

[120]  G. Lamas,et al.  Randomized Trials of Vitamin E in the Treatment and Prevention of Cardiovascular Disease , 2004 .

[121]  John M Colford,et al.  Systematic reviews and meta-analyses: an illustrated, step-by-step guide. , 2004, The National medical journal of India.

[122]  R. Derby,et al.  The relation between annular disruption on computed tomography scan and pressure-controlled diskography. , 2005, Archives of physical medicine and rehabilitation.

[123]  J. Ioannidis.  Contradicted and initially stronger effects in highly cited clinical research. , 2005, JAMA.

[124]  Pamela Royle,et al.  Systematic reviews of epidemiology in diabetes: finding the evidence , 2005, BMC medical research methodology.

[125]  David Moher,et al.  Language of publication restrictions in systematic reviews gave different results depending on whether the intervention was conventional or complementary. , 2005, Journal of clinical epidemiology.

[126]  Kouhei Akazawa,et al.  The effect of a multiple literature database search—A numerical evaluation in the domain of Japanese life science , 2005, Journal of Medical Systems.

[127]  S. Normand,et al.  Reader's guide to critical appraisal of cohort studies: 1. Role and design , 2005, BMJ : British Medical Journal.

[128]  Roberto Pastor-Barriuso,et al.  Meta-Analysis: High-Dosage Vitamin E Supplementation May Increase All-Cause Mortality , 2005, Annals of Internal Medicine.

[129]  C. Goldsmith,et al.  Medicinal and injection therapies for mechanical neck disorders. , 2015, The Cochrane database of systematic reviews.

[130]  G. Preminger,et al.  Clinical research and statistical methods in the urology literature. , 2005, The Journal of urology.

[131]  Gary H Lyman,et al.  The strengths and limitations of meta-analyses based on aggregate data , 2005, BMC Medical Research Methodology.

[132]  David J Samson,et al.  Challenges in Systematic Reviews of Diagnostic Technologies , 2005, Annals of Internal Medicine.

[133]  P. Austin,et al.  Reader's guide to critical appraisal of cohort studies: 2. Assessing potential for confounding , 2005, BMJ : British Medical Journal.

[134]  D. Cook,et al.  Evaluating Meta-analyses in the General Surgical Literature: A Critical Appraisal , 2005, Annals of surgery.

[135]  A. Oxman,et al.  Improving the use of research evidence in guideline development: 8. Synthesis and presentation of evidence , 2006, Health research policy and systems.

[136]  Albert G. Veldhuizen,et al.  Percutaneous vertebroplasty as a treatment for osteoporotic vertebral compression fractures: a systematic review , 2006, European Spine Journal.

[137]  Jeremy M Grimshaw,et al.  Does updating improve the methodological and reporting quality of systematic reviews? , 2006, BMC medical research methodology.

[138]  R. Taylor,et al.  Balloon Kyphoplasty and Vertebroplasty for Vertebral Compression Fractures: A Comparative Systematic Review of Efficacy and Safety , 2006, Spine.

[139]  John P.A. Ioannidis,et al.  Comparison of evidence on harms of medical interventions in randomized and nonrandomized studies , 2006, Canadian Medical Association Journal.

[140]  Brett Hanscom,et al.  Surgical vs nonoperative treatment for lumbar disk herniation: the Spine Patient Outcomes Research Trial (SPORT) observational cohort. , 2006, JAMA.

[141]  I. Coulter.  Evidence Summaries and Synthesis: Necessary but Insufficient Approach for Determining Clinical Practice of Integrated Medicine? , 2006, Integrative cancer therapies.

[142]  S. Baudouin,et al.  A systematic review of the quality of genetic association studies in human sepsis , 2006, Intensive Care Medicine.

[143]  S. Ferguson,et al.  Vertebroplasty and Kyphoplasty: A Systematic Review of 69 Clinical Studies , 2006, Spine.

[144]  J. Skinner,et al.  Surgical vs nonoperative treatment for lumbar disk herniation: the Spine Patient Outcomes Research Trial (SPORT): a randomized trial. , 2006, JAMA.

[145]  Total hip arthroplasty: is small incision better? , 2006, The Journal of arthroplasty.

[146]  Douglas G Altman,et al.  Reporting of noninferiority and equivalence randomized trials: an extension of the CONSORT statement. , 2006, JAMA.

[147]  D. Holmes,et al.  Deconstructing the evidence-based discourse in health sciences: truth, power and fascism. , 2006, International journal of evidence-based healthcare.

[148]  G. Guyatt,et al.  Grading strength of recommendations and quality of evidence in clinical guidelines: report from an american college of chest physicians task force. , 2006, Chest.

[149]  W. Browner,et al.  Designing Clinical Research , 2006 .

[150]  E. Carragee,et al.  A Gold Standard Evaluation of the “Discogenic Pain” Diagnosis as Determined by Provocative Discography , 2006, Spine.

[151]  A D Oxman,et al.  Randomisation to protect against selection bias in healthcare trials. , 2011, The Cochrane database of systematic reviews.

[152]  C. Argoff,et al.  Assessment: Use of epidural steroid injections to treat radicular lumbosacral pain , 2007, Neurology.

[153]  Howard S. Smith,et al.  Systematic Review of the Diagnostic Utility of Selective Nerve Root Blocks , 2007 .

[154]  P. Chin,et al.  Comparing pain reduction following kyphoplasty and vertebroplasty for osteoporotic vertebral compression fractures. , 2007, Pain physician.

[155]  Y. van der Graaf,et al.  Percutaneous vertebroplasty compared with optimal pain medication treatment: short-term clinical outcome of patients with subacute or chronic painful osteoporotic vertebral compression fractures. The VERTOS study. , 2007, AJNR. American journal of neuroradiology.

[156]  L. Manchikanti,et al.  Physician payment 2008 for interventionalists: current state of health care policy. , 2007, Pain physician.

[157]  S. Pocock,et al.  Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): Explanation and Elaboration , 2007, Annals of Internal Medicine [serial online].

[158]  Kurt P Spindler,et al.  How to Write a Systematic Review , 2007, Clinical orthopaedics and related research.

[159]  N. Sehgal,et al.  Systematic review of diagnostic utility of facet (zygapophysial) joint injections in chronic spinal pain: an update. , 2007, Pain physician.

[160]  S. Datta,et al.  Systematic review of effectiveness and complications of adhesiolysis in the management of chronic spinal pain: an update. , 2007, Pain physician.

[161]  P. Shekelle,et al.  Effect of supplemental vitamin E for the prevention and treatment of cardiovascular disease , 2004, Journal of General Internal Medicine.

[162]  Howard S. Smith,et al.  Interventional techniques: evidence-based practice guidelines in the management of chronic spinal pain. , 2007, Pain physician.

[163]  Matthias Egger,et al.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies , 2007, PLoS medicine.

[164]  Ian Shrier,et al.  Should meta-analyses of interventions include observational studies in addition to randomized controlled trials? A critical examination of underlying principles. , 2007, American journal of epidemiology.

[165]  V. Singh,et al.  Systematic review of discography as a diagnostic test for spinal pain: an update. , 2007, Pain physician.

[166]  J. Higgins,et al.  Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. , 2007, International journal of epidemiology.

[167]  M. Swiontkowski,et al.  Complex Limb Salvage or Early Amputation for Severe Lower-Limb Injury: A Meta-Analysis of Observational Studies , 2007, Journal of orthopaedic trauma.

[168]  M. Hotopf,et al.  Bias in psychiatric case–control studies , 2007, British Journal of Psychiatry.

[169]  Steven P. Cohen,et al.  Sacroiliac joint interventions: a systematic review. , 2007, Pain physician.

[170]  Salahadin Abdi,et al.  Epidural steroids in the management of chronic spinal pain: a systematic review. , 2007, Pain physician.

[171]  David Moher,et al.  Epidemiology and Reporting Characteristics of Systematic Reviews , 2007, PLoS medicine.

[172]  L. Manchikanti,et al.  Evidence-based interventional pain management: principles, problems, potential and applications. , 2007, Pain physician.

[173]  D. Altman,et al.  Commentary: strengthening the reporting of observational epidemiology the STROBE statement. , 2007, International journal of epidemiology.

[174]  L. Manchikanti,et al.  Issues in health care: interventional pain management at the crossroads. , 2007, Pain physician.

[175]  Howard S. Smith,et al.  Evidence-based medicine, systematic reviews, and guidelines in interventional pain management: Part 2: Randomized controlled trials. , 2008, Pain physician.

[176]  D. Moher,et al.  Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration. , 2008, Annals of internal medicine.

[177]  T. Shiga,et al.  Is operative delay associated with increased mortality of hip fracture patients? Systematic review, meta-analysis, and meta-regression , 2008, Canadian journal of anaesthesia = Journal canadien d'anesthesie.

[178]  L. Manchikanti.  Health care reform in the United States: radical surgery needed now more than ever. , 2008, Pain physician.

[179]  D. Richards.  Handsearching still a valuable element of the systematic review , 2008, Evidence-Based Dentistry.

[180]  P. Staats,et al.  Review of occupational medicine practice guidelines for interventional pain management and potential implications. , 2008, Pain physician.

[181]  Brett Hanscom,et al.  Surgical versus nonsurgical therapy for lumbar spinal stenosis. , 2008, The New England journal of medicine.

[182]  M. Nordin,et al.  Treatment of Neck Pain: Injections and Surgical Interventions: Results of the Bone and Joint Decade 2000–2010 Task Force on Neck Pain and Its Associated Disorders , 2008, Spine.

[183]  P Mills,et al.  The SQUIRE (Standards for QUality Improvement Reporting Excellence) guidelines for quality improvement reporting: explanation and elaboration , 2008, Quality & Safety in Health Care.

[184]  A. Marson,et al.  Overwhelming heterogeneity in systematic reviews of observational anti-epileptic studies , 2008, Epilepsy Research.

[185]  Philipp Dahm,et al.  How to use a systematic literature review and meta-analysis. , 2008, The Journal of urology.

[186]  David Moher,et al.  No consensus exists on search reporting methods for systematic reviews. , 2008, Journal of clinical epidemiology.

[187]  Laxmaiah Manchikanti,et al.  Evidence-based medicine, systematic reviews, and guidelines in interventional pain management, part I: introduction and general considerations. , 2008, Pain physician.

[188]  Anne Eisinga,et al.  Enhancing access to reports of randomized trials published world-wide – the contribution of EMBASE records to the Cochrane Central Register of Controlled Trials (CENTRAL) in The Cochrane Library , 2008, Emerging themes in epidemiology.

[189]  L. Manchikanti,et al.  A critical appraisal of 2007 American College of Occupational and Environmental Medicine (ACOEM) Practice Guidelines for Interventional Pain Management: an independent review utilizing AGREE, AMA, IOM, and other criteria. , 2008, Pain physician.

[190]  R. Chou.  Generating evidence on spinal cord stimulation for failed back surgery syndrome: not yet fully charged. , 2008, The Clinical journal of pain.

[191]  A. Oxman.  The Cochrane Collaboration in the 21st Century: Ten Challenges and One Reason Why They Must Be Met , 2008 .

[192]  R. Derby,et al.  Systematic review of lumbar provocation discography in asymptomatic subjects with a meta-analysis of false-positive rates. , 2008, Pain physician.

[193]  L. Manchikanti,et al.  Reassessment of evidence synthesis of occupational medicine practice guidelines for interventional pain management. , 2008, Pain physician.

[194]  Kristian Thorlund,et al.  Attention should be given to multiplicity issues in systematic reviews. , 2008, Journal of clinical epidemiology.

[195]  Youping Li,et al.  The reporting quality of meta-analyses improves: a random sampling study. , 2008, Journal of clinical epidemiology.

[196]  D. Moher,et al.  Issues related to the conduct of systematic reviews: a focus on the nutrition field. , 2008, The American journal of clinical nutrition.

[197]  Nikolaos A Patsopoulos,et al.  Bmc Medical Research Methodology Open Access Reporting of Human Genome Epidemiology (huge) Association Studies: an Empirical Assessment , 2022 .

[198]  A. Hoes,et al.  Poor quality of reporting confounding bias in observational intervention studies: a systematic review. , 2008, Annals of epidemiology.

[199]  M. Bhandari,et al.  Analysis of observational studies: a guide to understanding statistical methods. , 2009, The Journal of bone and joint surgery. American volume.

[200]  L. Manchikanti,et al.  Systematic review of the effectiveness of thermal annular procedures in treating discogenic low back pain. , 2009, Pain physician.

[201]  J. Ioannidis,et al.  Strengthening the reporting of genetic association studies (STREGA): an extension of the strengthening the reporting of observational studies in epidemiology (STROBE) statement. , 2009, Journal of clinical epidemiology.

[202]  L. Manchikanti,et al.  Critical evaluation of interventional pain management literature provides inaccurate conclusions. , 2009, The spine journal : official journal of the North American Spine Society.

[203]  Salahadin Abdi,et al.  Lumbar interlaminar epidural injections in managing chronic low back and lower extremity pain: a systematic review. , 2009, Pain physician.

[204]  Steven P. Cohen,et al.  Systematic review of lumbar discography as a diagnostic test for chronic low back pain. , 2009, Pain physician.

[205]  Howard S. Smith,et al.  Effectiveness of spinal endoscopic adhesiolysis in post lumbar surgery syndrome: a systematic review. , 2009, Pain physician.

[206]  S. Datta,et al.  Systematic review of diagnostic utility and therapeutic effectiveness of cervical facet joint interventions. , 2009, Pain physician.

[207]  R. Deyo,et al.  Overtreating Chronic Back Pain: Time to Back Off? , 2009, The Journal of the American Board of Family Medicine.

[208]  L. Manchikanti,et al.  Evidence-based medicine, systematic reviews, and guidelines in interventional pain management: part 5. Diagnostic accuracy studies. , 2009, Pain physician.

[209]  S. Datta,et al.  Systematic review of caudal epidural injections in the management of chronic low back pain. , 2009, Pain physician.

[210]  Howard S. Smith,et al.  Comprehensive review of therapeutic interventions in managing chronic spinal pain. , 2009, Pain physician.

[211]  R. Bluhm.  Some Observations on “Observational” Research , 2009, Perspectives in biology and medicine.

[212]  J. Ioannidis,et al.  The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration , 2009, Annals of Internal Medicine [serial online].

[213]  Howard S. Smith,et al.  Comprehensive review of neurophysiologic basis and diagnostic interventions in managing chronic spinal pain. , 2009, Pain physician.

[214]  Joshua H Levin,et al.  Prospective, double-blind, randomized placebo-controlled trials in interventional spine: what the highest quality literature tells us. , 2009, The spine journal : official journal of the North American Spine Society.

[215]  William Hollingworth,et al.  A randomized trial of vertebroplasty for osteoporotic spinal fractures. , 2009, The New England journal of medicine.

[216]  H. D. de Vet,et al.  Injection Therapy for Subacute and Chronic Low Back Pain: An Updated Cochrane Review , 2009, Spine.

[217]  Howard S. Smith,et al.  Comprehensive evidence-based guidelines for interventional techniques in the management of chronic spinal pain. , 2009, Pain physician.

[218]  Steven P. Cohen,et al.  Spinal cord stimulation for patients with failed back surgery syndrome: a systematic review. , 2009, Pain physician.

[219]  L. Manchikanti,et al.  Automated percutaneous lumbar discectomy for the contained herniated lumbar disc: a systematic assessment of evidence. , 2009, Pain physician.

[220]  Nicolette de Keizer,et al.  STARE-HI -Statement on Reporting of Evaluation Studies in Health Informatics , 2009, Yearbook of Medical Informatics.

[221]  Howard S. Smith,et al.  Systematic review of therapeutic lumbar transforaminal epidural steroid injections. , 2009, Pain physician.

[222]  Christine Y. Lu,et al.  Observational studies: a review of study designs, challenges and strategies to reduce confounding , 2009, International journal of clinical practice.

[223]  Steven P. Cohen,et al.  Systematic review of cervical discography as a diagnostic test for chronic spinal pain. , 2009, Pain physician.

[224]  J. Groopman,et al.  Keeping the patient in the equation--humanism and health care reform. , 2009, The New England journal of medicine.

[225]  S. Datta,et al.  Systematic assessment of diagnostic accuracy and therapeutic utility of lumbar facet joint interventions. , 2009, Pain physician.

[226]  G. Gartlehner,et al.  [Kyphoplasty and vertebroplasty for the management of osteoporotic vertebral compression fractures: a systematic review]. , 2009, Der Orthopade.

[227]  J. Weinstein.  Balancing science and informed choice in decisions about vertebroplasty. , 2009, The New England journal of medicine.

[228]  James N Weinstein,et al.  Surgical compared with nonoperative treatment for lumbar degenerative spondylolisthesis. four-year results in the Spine Patient Outcomes Research Trial (SPORT) randomized and observational cohorts. , 2009, The Journal of bone and joint surgery. American volume.

[229]  Howard S. Smith,et al.  Systematic review of the role of sedation in diagnostic spinal interventional techniques. , 2009, Pain physician.

[230]  Steven P. Cohen,et al.  Evaluation of sacroiliac joint interventions: a systematic appraisal of the literature. , 2009, Pain physician.

[231]  V. Singh,et al.  Systematic Review of the Effectiveness of Cervical Epidurals in the Management of Chronic Neck Pain , 2010 .

[232]  Howard S. Smith,et al.  Systematic review of intrathecal infusion systems for long-term management of chronic non-cancer pain. , 2009, Pain physician.

[233]  N. Simunovic,et al.  Methodological issues in systematic reviews and meta-analyses of observational studies in orthopaedic research. , 2009, The Journal of bone and joint surgery. American volume.

[234]  Howard S. Smith,et al.  Analysis of growth of interventional techniques in managing chronic pain in the Medicare population: a 10-year evaluation from 1997 to 2006. , 2009, Pain physician.

[235]  Howard S. Smith,et al.  Systematic review of percutaneous adhesiolysis and management of chronic low back pain in post lumbar surgery syndrome. , 2009, Pain physician.

[236]  J. Stockman.  How Quickly Do Systematic Reviews Go Out of Date? A Survival Analysis , 2009 .

[237]  T. Greene.  Randomized and observational studies in nephrology: how strong is the evidence? , 2009, American journal of kidney diseases : the official journal of the National Kidney Foundation.

[238]  Evidence-based or evidence-biased: the need to re-appraise and re-align levels of information with stakeholder values. , 2009, Pain physician.

[239]  R. Buchbinder,et al.  A randomized trial of vertebroplasty for painful osteoporotic vertebral fractures. , 2009, The New England journal of medicine.

[240]  Howard S. Smith,et al.  Evidence-based medicine, systematic reviews, and guidelines in interventional pain management: part 4: observational studies. , 2009, Pain physician.

[241]  Philip S. Wang,et al.  Invited Commentary: Assessing treatment effects by using observational analyses--opportunities and limitations. , 2009, American journal of epidemiology.

[242]  J. Ioannidis,et al.  STrengthening the REporting of Genetic Association Studies (STREGA)— An Extension of the STROBE Statement , 2009, PLoS medicine.

[243]  Constantine Gatsonis,et al.  Analysing and Presenting Results , 2010 .

[244]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. , 2010, International journal of surgery.

[245]  D. Sackett.  Evidence-Based Medicine: How to Practice and Teach EBM , 2018 .