Conflict between guideline methodologic quality and recommendation validity: a potential problem for practitioners.

BACKGROUND It is not clear if good methodologic quality in current practice guidelines necessarily leads to more valid recommendations, i.e., those that are supported with consistent research evidence or, when evidence is conflicting or lacking, with sufficient consensus among the guideline development team. To help clarify this issue, we assessed whether there is a link between methodologic quality and recommendation validity in practice guidelines for the use of laboratory tests in the management of patients with non-small cell lung cancer (NSCLC). METHODS We conducted a systematic review of data on laboratory tests in NSCLC published in English or in French within the last 10 years and retrieved 11 practice guidelines for the use of these tests. The guidelines were critically appraised and scored for methodologic quality and recommendation validity based on the Appraisal of Guidelines Research and Evaluation (AGREE) criteria and on the systematic review. RESULTS Overall, these 11 guidelines had considerable shortcomings in methodologic quality and, to a lesser extent, in recommendation validity. Practice guidelines with the best methodologic quality were not necessarily the most valid in their recommendations, and conversely. CONCLUSIONS Poor methodologic quality and lack of recommendation validity in laboratory medicine call for methodologic standards of guideline development and for international collaboration of guideline development agencies. We advise readers of guidelines to critically evaluate the methods used as well as the content of the recommendations before adopting them for use in practice.

[1]  M Paesmans,et al.  CYFRA 21-1 is a prognostic determinant in non-small-cell lung cancer: results of a meta-analysis in 2063 patients , 2004, British Journal of Cancer.

[2]  D. Seligson,et al.  Clinical Chemistry , 1965, Bulletin de la Societe de chimie biologique.

[3]  J. Watine.  Are Laboratory Investigations Recommended in Current Medical Practice Guidelines Supported by Available Evidence? , 2002, Clinical chemistry and laboratory medicine.

[4]  K. Hannes,et al.  A systematic review of appraisal tools for clinical practice guidelines: multiple similarities and one common deficit. , 2005, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[5]  K. Khunti,et al.  Diagnosis of heart failure in primary care: an assessment of international guidelines. , 2001, The British journal of general practice : the journal of the Royal College of General Practitioners.

[6]  W. Stalman,et al.  Nonsteroidal antiinflammatory drugs or acetaminophen for osteoarthritis of the hip or knee? A systematic review of evidence and guidelines. , 2004, The Journal of rheumatology.

[7]  D. Moher,et al.  The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials , 2001, The Lancet.

[8]  J. Lagrange,et al.  Standards, Options et Recommandations 2000 pour la prise en charge des patients atteints d'un cancer bronchopulmonaire non à petites cellules (rapport abrégé) , 2003 .

[9]  W. Jahn,et al.  An independent assessment of chiropractic practice guidelines. , 2003, Journal of manipulative and physiological therapeutics.

[10]  P. Shekelle,et al.  Validity of the Agency for Healthcare Research and Quality clinical practice guidelines: how quickly do guidelines become outdated? , 2001, JAMA.

[11]  Clinical practice guidelines for the treatment of unresectable non-small-cell lung cancer. Adopted on May 16, 1997 by the American Society of Clinical Oncology. , 1997, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[12]  F. Paccaud,et al.  Screening for gestational diabetes: variation in guidelines. , 2000, European journal of obstetrics, gynecology, and reproductive biology.

[13]  C. Bombardier,et al.  Quality of Primary Care Guidelines for Acute Low Back Pain , 2004, Spine.

[14]  M. Emberton,et al.  A review of guidelines on benign prostatic hyperplasia and lower urinary tract symptoms: are all guidelines the same? , 2003, BJU international.

[15]  D. Mccrory,et al.  Assessment of the scope and quality of clinical practice guidelines in lung cancer. , 2003, Chest.

[16]  G. Kalemkerian,et al.  Small cell lung cancer , 2010, Seminars in Respiratory and Critical Care Medicine.

[17]  V. Preedy,et al.  Scottish Intercollegiate Guidelines Network , 2010 .

[18]  Tumour markers in lung cancer: EGTM recommendations. European Group on Tumour Markers. , 1999, Anticancer research.

[19]  N. Klazinga,et al.  Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project [AGREE] , 2003 .

[20]  J. Burgers,et al.  Beyond the evidence in clinical guidelines , 2004, The Lancet.

[21]  M. Mäkelä,et al.  Practice guidelines in Finland: availability and quality. , 1997, Quality in health care : QHC.

[22]  P. Shekelle,et al.  Is the methodological quality of guidelines declining in the US? Comparison of the quality of US Agency for Health Care Policy and Research (AHCPR) guidelines with those published subsequently , 2003, Quality & safety in health care.

[23]  G. Guyatt,et al.  Grading quality of evidence and strength of recommendations , 2004, BMJ : British Medical Journal.

[24]  I D Graham,et al.  The management of hypertension in Canada: a review of current guidelines, their shortcomings and implications for the future. , 2001, CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne.

[25]  I. Durand-Zaleski,et al.  Reducing unnecessary laboratory use with new test request form: example of tumour markers , 1993, The Lancet.

[26]  W. Alberts.  Lung Cancer Guidelines: Introduction , 2003 .

[27]  R. Thomson,et al.  Clinical guidelines: quantity without quality. , 1997, Quality in health care : QHC.

[28]  G. Baum.  Textbook of Pulmonary Diseases , 1989 .

[29]  BTS guidelines Guidelines on the selection of patients with lung cancer for surgery British Thoracic Society and Society of Cardiothoracic Surgeons of Great Britain and Ireland Working Party , 2000 .

[30]  M. Baltzan.  Assessing the quality of clinical practice guidelines. , 2002, CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne.

[31]  J. Watine.  Prognostic evaluation of primary non-small cell lung carcinoma patients using biological fluid variables. A systematic review , 2000, Scandinavian journal of clinical and laboratory investigation.

[32]  L. Freedman,et al.  The future of prognostic factors in outcome prediction for patients with cancer , 1992, Cancer.

[33]  T Philip,et al.  International assessment of the quality of clinical practice guidelines in oncology using the Appraisal of Guidelines and Research and Evaluation Instrument. , 2004, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[34]  J. Grimshaw,et al.  The quantity and quality of clinical practice guidelines for the management of depression in primary care in the UK. , 1999, The British journal of general practice : the journal of the Royal College of General Practitioners.

[35]  J. Watine,et al.  Anemia as an independent prognostic factor for survival in patients with cancer , 2002, Cancer.

[36]  David Moher,et al.  The CONSORT statement: revised recommendations for improving the quality of reports of parallel group randomized trials , 2001, Annals of Internal Medicine.

[37]  R. Ginsberg.  Continuing controversies in staging NSCLC: an analysis of the revised 1997 staging system. , 1998, Oncology.

[38]  J. Watine,et al.  Faut-il suivre les sociétés savantes de pneumologie dans leurs recommandations concernant l'utilisation des marqueurs tumoraux sériques pour l'évaluation pronostique des cancers bronchiques primitifs non à petites cellules ? , 1999 .

[39]  Sverre Sandberg,et al.  Evidence-based guidelines in laboratory medicine: principles and methods. , 2004, Clinical chemistry.

[40]  R. Grol,et al.  Inside guidelines: comparative analysis of recommendations and evidence in diabetes guidelines from 13 countries. , 2002, Diabetes care.

[41]  G. Feder,et al.  Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project , 2003, Quality & safety in health care.

[42]  T. British,et al.  BTS guidelines: guidelines on the selection of patients with lung cancer for surgery. , 2001, Thorax.

[43]  J. Ward,et al.  Why we need guidelines for guidelines: a study of the quality of clinical practice guidelines in Australia , 1996, The Medical journal of Australia.

[44]  P. Godeau.  Conférence de consensus , 2000 .

[45]  M. Mayo-Smith,et al.  Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature. , 1999, JAMA.

[46]  I. Graham,et al.  Systematic assessment of the quality of osteoporosis guidelines , 2002, BMC musculoskeletal disorders.

[47]  M. Offringa,et al.  Quality of Evidence-Based Pediatric Guidelines , 2005, Pediatrics.

[48]  Aniruddha M. Deshpande,et al.  Standardized Reporting of Clinical Practice Guidelines: A Proposal from the Conference on Guideline Standardization , 2003, Annals of Internal Medicine.

[49]  P G Shekelle,et al.  Are nonspecific practice guidelines potentially harmful? A randomized comparison of the effect of nonspecific versus specific guidelines on physician decision making. , 2000, Health services research.

[50]  F. McAlister,et al.  What is the quality of drug therapy clinical practice guidelines in Canada? , 2001, CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne.

[51]  V. Devita,et al.  Cancer : Principles and Practice of Oncology , 1982 .

[52]  W. Hanbury.  Bronchogenic Carcinoma in Women , 1964, Thorax.

[53]  A. Horvath,et al.  Quality of Guidelines for the Laboratory Management of Diabetes Mellitus , 2005, Scandinavian journal of clinical and laboratory investigation. Supplementum.

[54]  Alessandro Liberati,et al.  Practice guidelines developed by specialty societies: the need for a critical appraisal , 2000, The Lancet.

[55]  M. Weinbren.  Bronchogenic carcinoma. , 1953, South African medical journal = Suid-Afrikaanse tydskrif vir geneeskunde.