Benefits and risks of drug treatments: how to combine the best evidence on benefits with the best data about adverse effects.

THE CENTRAL THEME OF THE INSTITUTE OF MEDICINE report on the US drug safety system was the need for a life cycle approach to drug evaluation: both the benefits and the risks need to be evaluated and integrated during the entire market life of a drug. The Food and Drug Administration Amendments Act of 2007 also called on the agency to improve its methods of communicating risks and benefits to patients and physicians. The Institute of Medicine recommendation to “develop and continually improve a systematic approach to risk-benefit analysis for use throughout the [Food and Drug Administration] in the preapproval and postapproval settings” specifically acknowledges the need for and the challenges of the development of new methods of combining evidence about risks and benefits. Information that combines the best evidence on benefits with the best data on risks is also needed for daily clinical practice. Whenever a patient and physician decide on a particular course of treatment, they do so because they expect that the likely benefits will exceed potential harms. For the benefits of drug treatments, they often have authoritative sources to provide information: randomized trials and systematic reviews and meta-analyses of such trials. For adverse effects, the situation is different. Given the average duration of randomized trials (often months to 1 or 2 years) and the average number of patients in randomized trials (often dozens to a few hundred), such trials are at most able to detect and quantify frequent adverse events that occur only early during treatment. Moreover, the adverse effect has to be known beforehand or anticipated to be recorded systematically in the trials. The study population in trials, which often includes young persons with a single diagnosis and without concurrent disease, is often not representative of those who will eventually use the drug in the community. The situation does not much improve in meta-analysis: the typical meta-analysis of randomized trials covers 1000 to 2500 individuals, only half of whom will have taken the new drug. The sample size precludes good quantification of adverse effects unless they occur at least with a frequency of about 1 per 200 person-years. Meta-analyses of trials do not solve the problem of late adverse effects or the problem of the narrow population included in the trials. The information on harms from trials is incomplete, and the possibility of using and combining such information across trials in systematic reviews is limited. Thus, to understand the full spectrum of adverse effects—those that occur late, that were not known beforehand, and that are rare but nevertheless serious—and to be able to investigate the true incidence of known adverse effects in circumstances of actual prescribing, well-designed observational studies will always be necessary. It follows that systematic reviews of drug treatments must include not only the results of randomized trials on benefits but also evidence from observational research on harms. Although the idea to turn to observational data for evidence may be a surprise, the idea makes sense in view of what randomization does and what randomized trials are good at doing. Randomized trials are uniquely superior to evaluate the benefits, the intended or hoped-for effects of treatments. The random allocation mechanism is able to overcome the strong tendency of physicians to selectively prescribe treatments based on the perceived prognosis and likely outcomes of patients. Although prescriptions that are tailored to individual prognoses constitute desired medical practice, this effort also makes it difficult to compare the benefits of different treatments because patients who receive different treatments will have different prognoses, called “confounding by indication.” In general, data from routine daily practice can therefore not be used to assess the benefits of treatment, especially for the comparison between users and nonusers of a drug.