The ISPOR Good Practices for Quality Improvement of Cost-Effectiveness Research Task Force Report.

OBJECTIVES The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Health Science Policy Council recommended and the ISPOR Board of Directors approved the formation of a Task Force to critically examine the major issues related to Quality Improvement in Cost-effectiveness Research (QICER). The Council's primary recommendation for this Task Force was that it should report on the quality of cost-effectiveness research and make recommendations to facilitate the improvement of pharmacoeconomics and health outcomes research and its use in stimulating better health care and policy. Task force members were knowledgeable and experienced in medicine, pharmacy, biostatistics, health policy and health-care decision-making, biomedical knowledge transfer, health economics, and pharmacoeconomics. They were drawn from industry, academia, consulting organizations, and advisors to governments and came from Japan, the Netherlands, Canada and the United States. METHODS Face-to-face meetings of the Task Force were held at ISPOR North American and European meetings and teleconferences occurred every few months. Literature reviews and surveys were conducted and the first preliminary findings presented at an open forum at the May 2008 ISPOR meeting in Toronto. The final draft report was circulated to the expert reviewer group and then to the entire membership for comment. The draft report was posted on the ISPOR Web site in April 2009. All formal comments received were posted to the association Web site and presented for discussion at the Task Force forum during the ISPOR 14th Annual International Meeting in May 2009. Comments and feedback from the forums, reviewers and membership were considered in the final report. Once Task Force consensus was reached, the article was submitted to Value in Health. CONCLUSIONS The QICER Task Force recommends that ISPOR implement the following: * With respect to CER guidelines, that ISPOR promote harmonization of guidelines, allowing for differences in application, regional needs and politics; evaluate available instruments or promote development of a new one that will allow standardized quantification of the impact of CER guidelines on the quality of CER studies; report periodically on those countries or regions that have developed guidelines; periodically evaluate the quality of published studies (those journals with CER guidances) or those submitted to decision-making bodies (as public transparency increases). * With respect to methodologies, that ISPOR promote publication of methodological guidelines in more applied journals in more easily understandable format to transfer knowledge to researchers who need to apply more rigorous methods; promote full availability of models in electronic format to combat space restrictions in hardcopy publications; promote consistency of methodological review for all CER studies; promote adoption of explicit best practices guidelines among regulatory and reimbursement authorities; periodically update all ISPOR Task Force reports; periodically review use of ISPOR Task Force guidelines; periodically report on statistical and methodological challenges in HE; evaluate periodically whether ISPOR's methodological guidelines lead to improved quality; and support training and knowledge transfer of rigorous CER methodologies to researchers and health care decision-makers. * With respect to publications, that ISPOR develop standard CER guidances to which journals will be able to refer their authors and their reviewers; lobby to establish these guidances within the International Committee for Medical Journal Editors (ICMJE) Requirements to which most journals refer in their Author Instructions; provide support in terms of additional reviewer expertise to those journals lacking appropriate reviewers; periodically report on journals publishing CER research; periodically report on the quality of CER publications; and support training and knowledge transfer of the use of these guidelines to researchers and reviewers. * With respect to evidence-based health-care decision-making, that ISPOR recognize at its annual meetings those countries/agencies/private companies/researchers using CER well, and those practitioners and researchers supporting good patient use of CER in decision-making; and promote public presentation of case studies of applied use of CER concepts or guidelines.

[1]  A H Briggs,et al.  Power and Sample Size Calculations for Stochastic Cost-Effectiveness Analysis , 1998, Medical decision making : an international journal of the Society for Medical Decision Making.

[2]  E. Feuer,et al.  Estimating medical costs from incomplete follow-up data. , 1997, Biometrics.

[3]  L. Mantovani,et al.  Guidelines for Economic Evaluations in Italy: Recommendations from The Italian Group of Pharmacoeconomic Studies , 2001 .

[4]  Sue Eckstein,et al.  Good research practice , 2003 .

[5]  B. O'brien,et al.  Country specific cost comparisons from multinational clinical trials using empirical Bayesian shrinkage estimation: the Canadian ASSENT-3 economic analysis. , 2005, Health economics.

[6]  W. Manning,et al.  Estimating Log Models: To Transform or Not to Transform? , 1999, Journal of health economics.

[7]  M. Gold,et al.  Panel on cost-effectiveness in health and medicine. , 1996, Medical care.

[8]  R. Willke,et al.  Good research practices for cost-effectiveness analysis alongside clinical trials: the ISPOR RCT-CEA Task Force report. , 2005, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[9]  P. McCullagh,et al.  Generalized Linear Models , 1992 .

[10]  J. Graf von der Schulenburg,et al.  The influence of economic evaluation studies on decision making. A European survey. The EUROMET group. , 2000, Health policy.

[11]  S. Stearns,et al.  Estimating confidence intervals for cost-effectiveness ratios: an example from a randomized trial. , 1996, Statistics in medicine.

[12]  W. Manning,et al.  The logged dependent variable, heteroscedasticity, and the retransformation problem. , 1998, Journal of health economics.

[13]  John C. Hershey,et al.  Carrier Screening for Cystic Fibrosis , 1998, Medical decision making : an international journal of the Society for Medical Decision Making.

[14]  R. Grieve,et al.  Using multilevel models for assessing the variability of multinational resource use and cost data. , 2005, Health economics.

[15]  G. Lu,et al.  Combination of direct and indirect evidence in mixed treatment comparisons , 2004, Statistics in medicine.

[16]  B. O'brien,et al.  The death of cost-minimization analysis? , 2001, Health economics.

[17]  B. O'brien,et al.  Cost–effectiveness analysis for multinational clinical trials , 2005, Statistics in medicine.

[18]  M. Drummond,et al.  Do health-care decision makers find economic evaluations useful? The findings of focus group research in UK health authorities. , 2002, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[19]  Simon G Thompson,et al.  Uncertainty and validation of health economic decision models. , 2009, Health economics.

[20]  Jeroen P Jansen,et al.  Bayesian meta-analysis of multiple treatment comparisons: an introduction to mixed treatment comparisons. , 2008, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[21]  P. McCullagh,et al.  Generalized Linear Models, 2nd Edn. , 1990 .

[22]  Milton C. Weinstein,et al.  The role of cost-effectiveness analysis in health and medicine. Panel on Cost-Effectiveness in Health and Medicine. , 1996 .

[23]  M. Kelly Addressing the issues , 2002 .

[24]  M. Drummond,et al.  Evaluating the BMJ guidelines for economic submissions: prospective audit of economic submissions to BMJ and The Lancet. , 1998, JAMA.

[25]  N. Muszbek,et al.  Methodological guidelines for conducting economic evaluation of healthcare interventions in Hungary: a Hungarian proposal for methodology standards , 2002, The European Journal of Health Economics.

[26]  F. Antoñanzas,et al.  Economic Analysis of Health Technologies and Programmes , 1995, PharmacoEconomics.

[27]  G. Brown,et al.  Analysis of costs. , 2000, Archives of ophthalmology.

[28]  C. Bell,et al.  The Quality of Reporting in Published Cost-Utility Analyses, 19761997 , 2000, Annals of Internal Medicine.

[29]  M A Koopmanschap,et al.  Analysis of costs and cost-effectiveness in multinational trials. , 2001, Health policy.

[30]  Dan Greenberg,et al.  Quality of Abstracts of Papers Reporting Original Cost-Effectiveness Analyses , 2005, Medical decision making : an international journal of the Society for Medical Decision Making.

[31]  M. Gail,et al.  Testing for qualitative interactions between treatment effects and patient subsets. , 1985, Biometrics.

[32]  Andrew R Willan,et al.  Regression methods for cost‐effectiveness analysis with censored data , 2005, Statistics in medicine.

[33]  J. Schulenburg The Influence of Economic Evaluation Studies on Health Care Decision-Making: A European Survey , 2000 .

[34]  D. Lin,et al.  Proportional Means Regression for Censored Medical Costs , 2000, Biometrics.

[35]  J. Hay,et al.  Examining the Value and Quality of Health Economic Analyses: Implications of Utilizing the QHES , 2003, Journal of managed care pharmacy : JMCP.

[36]  Deborah M Caldwell,et al.  Simultaneous comparison of multiple treatments: combining direct and indirect evidence , 2005, BMJ : British Medical Journal.

[37]  D. Lin,et al.  Regression analysis of incomplete medical cost data , 2003, Statistics in medicine.

[38]  M J Al,et al.  Sample size calculation in economic evaluations. , 1998, Health economics.

[39]  K Claxton,et al.  The irrelevance of inference: a decision-making approach to the stochastic evaluation of health care technologies. , 1999, Journal of health economics.

[40]  Anthony O'Hagan,et al.  Calculating Partial Expected Value of Perfect Information via Monte Carlo Sampling Algorithms , 2007, Medical decision making : an international journal of the Society for Medical Decision Making.

[41]  Morris Meisner,et al.  Power and Sample Size in Cost- Effectiveness Analysis , 1999, Medical decision making : an international journal of the Society for Medical Decision Making.

[42]  Katie Lahey,et al.  Raising the Quality , 2008 .

[43]  E. Walter,et al.  [Guidelines for health-economic evaluations in Austria]. , 2006, Wiener medizinische Wochenschrift.

[44]  D G Altman,et al.  Transfer of technology from statistical journals to the biomedical literature. Past trends and future predictions. , 1994, JAMA.

[45]  S. Sullivan,et al.  Incorporating clinical outcomes and economic consequences into drug formulary decisions: a practical approach. , 1999, The American journal of managed care.

[46]  D Polsky,et al.  Confidence intervals for cost-effectiveness ratios: a comparison of four methods. , 1997, Health economics.

[47]  Milton C. Weinstein,et al.  Recommendations for Reporting Cost-effectiveness Analyses , 1996 .

[48]  J. Hutton,et al.  Use of economic evaluation in decision making: what needs to change? , 2002, Value in Health.

[49]  M. Drummond,et al.  A rational framework for decision making by the National Institute For Clinical Excellence (NICE) , 2002, The Lancet.

[50]  A E Ades,et al.  Expected Value of Sample Information Calculations in Medical Decision Modeling , 2004, Medical decision making : an international journal of the Society for Medical Decision Making.

[51]  M. Weinstein,et al.  Recommendations of the Panel on Cost-effectiveness in Health and Medicine. , 1996, JAMA.

[52]  Anthony O'Hagan,et al.  On estimators of medical costs with censored data. , 2004, Journal of health economics.

[53]  A. Mcguire,et al.  Estimating medical care costs under conditions of censoring. , 2004, Journal of health economics.

[54]  P Fenn,et al.  The Analysis of Censored Treatment Cost Data in Economic Evaluation , 1995, Medical care.

[55]  A. L. Hillman,et al.  Economic Analysis of Health Care Technology: A Report on Principles , 1995, Annals of Internal Medicine.

[56]  A H Briggs,et al.  Quantifying stochastic uncertainty and presenting results of cost-effectiveness analyses , 2001, Expert review of pharmacoeconomics & outcomes research.

[57]  M. Drummond,et al.  Health Care Technology: Effectiveness, Efficiency and Public Policy@@@Methods for the Economic Evaluation of Health Care Programmes , 1988 .

[58]  Nigel Rice,et al.  Cost-Effectiveness Analysis Using Data from Multinational Trials: The Use of Bivariate Hierarchical Modeling , 2007, Medical decision making : an international journal of the Society for Medical Decision Making.

[59]  S. Thompson,et al.  Analysis of cost data in randomized trials: an application of the non-parametric bootstrap. , 2000, Statistics in medicine.

[60]  N. Olchanski,et al.  Growth and quality of the cost-utility literature, 1976-2001. , 2005, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[61]  J. Seymour,et al.  Raising the quality of cost-utility analyses: lessons learnt and still to learn. , 1999, Health policy.

[62]  John Enos,et al.  TRANSFER OF TECHNOLOGY , 1989 .

[63]  Péter Jacsó,et al.  Quality of abstracts , 1993 .

[64]  Joseph C Gardiner,et al.  Longitudinal analysis of censored medical cost data. , 2006, Health economics.

[65]  B. McNeil,et al.  Probabilistic Sensitivity Analysis Using Monte Carlo Simulation , 1985, Medical decision making : an international journal of the Society for Medical Decision Making.

[66]  J Hjelmgren,et al.  Health economic guidelines--similarities, differences and some implications. , 2001, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[67]  P W Lavori,et al.  A multiple imputation strategy for clinical trials with truncation of patient data. , 1995, Statistics in medicine.

[68]  A E Ades,et al.  A chain of evidence with mixed comparisons: models for multi‐parameter synthesis and consistency of evidence , 2003, Statistics in medicine.

[69]  Anastasios A. Tsiatis,et al.  Estimating medical costs with censored data , 2000 .

[70]  Daniel Polsky,et al.  Economic Evaluation in Clinical Trials , 2007 .

[71]  Jalpa A Doshi,et al.  Analyses of cost data in economic evaluations conducted alongside randomized controlled trials. , 2006, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[72]  R. Stanford,et al.  The National Health and Medical Research Council , 1937 .

[73]  P. Haywood,et al.  How to compare the costs and benefits: evaluation of the economic evidence , 2001 .

[74]  M J Al,et al.  Costs, effects and C/E-ratios alongside a clinical trial. , 1994, Health economics.

[75]  A A Stinnett,et al.  Net Health Benefits , 1998, Medical decision making : an international journal of the Society for Medical Decision Making.

[76]  Jan B Oostenbrink,et al.  The analysis of incomplete cost data due to dropout. , 2005, Health economics.

[77]  M. Drummond,et al.  The lag between effectiveness and cost-effectiveness evidence of new drugs , 2003, The European Journal of Health Economics, formerly: HEPAC.

[78]  Robert L Strawderman,et al.  Flexible hazard regression modeling for medical cost data. , 2002, Biostatistics.

[79]  S. M. Gore,et al.  The Lancet's statistical review process: areas for improvement by authors , 1992, The Lancet.

[80]  Sean D Sullivan,et al.  Quality of clinical and economic evidence in dossier formulary submissions. , 2007, The American journal of managed care.

[81]  S G Thompson,et al.  Analysis and interpretation of cost data in randomised controlled trials: review of published studies , 1998, BMJ.

[82]  D. Lin,et al.  Linear regression analysis of censored medical costs. , 2000, Biostatistics.

[83]  S. Hill,et al.  Problems with the interpretation of pharmacoeconomic analyses: a review of submissions to the Australian Pharmaceutical Benefits Scheme. , 2000, JAMA.

[84]  M C Hornbrook,et al.  Modeling risk using generalized linear models. , 1999, Journal of health economics.

[85]  S. T. Buckland,et al.  An Introduction to the Bootstrap. , 1994 .

[86]  Richard D Riley,et al.  Evidence synthesis combining individual patient data and aggregate data: a systematic review identified current practice and possible methods. , 2007, Journal of clinical epidemiology.

[87]  J. Heyse,et al.  A regression-based method for estimating mean treatment cost in the presence of right-censoring. , 2000, Biostatistics.

[88]  Alex J. Sutton,et al.  Methods for Meta-Analysis in Medical Research , 2000 .

[89]  Kimbroe J. Carter,et al.  Treatment of Early-Stage Breast Cancer in the Elderly , 1998, Medical decision making : an international journal of the Society for Medical Decision Making.

[90]  P. Trueman,et al.  Research into the use of health economics in decision making in the United Kingdom--Phase II. Is health economics 'for good or evil'? , 1999, Health policy.

[91]  Michael Drummond,et al.  Assessing the appropriateness of combining economic data from multinational clinical trials , 2003, Statistics in medicine.

[92]  Ruth Etzioni,et al.  Estimating the costs attributable to a disease with application to ovarian cancer. , 1996, Journal of clinical epidemiology.

[93]  Sean D Sullivan,et al.  Development and Validation of a Grading System for the Quality of Cost-Effectiveness Studies , 2003, Medical care.

[94]  P. Ball For good or evil , 2005 .

[95]  R. Robinson Economic Evaluation and Health Care Cost-effectiveness analysis , 2006 .

[96]  Milton C Weinstein,et al.  Principles of good practice for decision analytic modeling in health-care evaluation: report of the ISPOR Task Force on Good Research Practices--Modeling Studies. , 2003, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[97]  Bernie J. O'Brien,et al.  In Search of Power and Significance: Issues in the Design and Analysis of Stochastic Cost-Effectiveness Studies in Health Care , 1994, Medical care.

[98]  Karl Claxton,et al.  Characterizing structural uncertainty in decision analytic models: a review and application of methods. , 2009, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[99]  John Cairns,et al.  Multilevel models for estimating incremental net benefits in multinational studies. , 2007, Health economics.

[100]  M C Weinstein,et al.  Recommendations for reporting cost-effectiveness analyses. Panel on Cost-Effectiveness in Health and Medicine. , 1996, JAMA.

[101]  A R Willan,et al.  Confidence intervals for cost-effectiveness ratios: an application of Fieller's theorem. , 1996, Health economics.

[102]  M. Weinstein,et al.  The role of cost-effectiveness analysis in health and medicine. Panel on Cost-Effectiveness in Health and Medicine. , 1996, JAMA.

[103]  Richard Grieve,et al.  Addressing the issues that arise in analysing multicentre cost data, with application to a multinational study. , 2006, Journal of health economics.

[104]  R Robinson,et al.  Costs and cost-minimisation analysis. , 1993, BMJ.

[105]  T O Jefferson,et al.  Guidelines for authors and peer reviewers of economic submissions to the BMJ , 1996, BMJ.

[106]  Evelyn Walter,et al.  Guidelines zur gesundheitsökonomischen Evaluation Konsenspapier , 2006, Wiener Medizinische Wochenschrift.

[107]  A R Willan,et al.  Sample size and power issues in estimating incremental cost-effectiveness ratios from clinical trials data. , 1999, Health economics.

[108]  S. Thompson,et al.  How should cost data in pragmatic randomised trials be analysed? , 2000, BMJ : British Medical Journal.

[109]  R. Willke,et al.  Estimating country-specific cost-effectiveness from multinational clinical trials. , 1998, Health economics.

[110]  N. Laird,et al.  Meta-analysis in clinical trials. , 1986, Controlled clinical trials.

[111]  J. Gray,et al.  The NHS economic evaluation database , 1997 .

[112]  M. Sculpher,et al.  Assessing generalisability by location in trial-based cost-effectiveness analysis: the use of multilevel models. , 2005, Health economics.

[113]  M. Drummond,et al.  Economic Evaluation in Health Care: Merging Theory with Practice , 2002 .

[114]  Andrew M. Jones,et al.  Multilevel models and health economics. , 1997, Health economics.

[115]  Dan Greenberg,et al.  Bias in published cost effectiveness studies: systematic review , 2006, BMJ : British Medical Journal.

[116]  L. Erickson,et al.  Evidence and Value: Impact on DEcisionMaking – the EVIDEM framework and potential applications , 2008, BMC health services research.