Missing Data in Longitudinal Clinical Trials Part A: Design and Conceptual Issues.

There are numerous types of missing data that can occur in clinical trials. Some types of missing data cannot be prevented and are beyond the research team's control. For example, a patient may relocate and be unavailable for an assessment. Other types occur because the research team actually designs the study to generate incomplete data. For example, it may not be cost-effective to obtain a full diagnostic assessment on each subject. Instead, an inexpensive screening tool is used to assess everyone. All screen positives, plus a random sample of screen negatives, are then given the full diagnostic. The incomplete diagnostic data on the majority of those screened negative are taken into account in assessing treatment effects.1-3 These types of data are missing by design, and there are statistical procedures to handle such planned missing data if designed appropriately. The third type occurs because of a faulty design plan by the research team. In this case, data are lost because a less than adequate protocol is followed. For example, if no screen negatives are given the full diagnostic, it will not be possible to ascertain the false negatives. As another example, we will see that serious missing data problems result from a trial design protocol that stops any further longitudinal assessment once a patient stops adhering to his or her assigned treatment. This article discusses how to avoid such faulty designs and how to design studies that can improve the design efficiency by using intentionally incomplete data. It also discusses how some analytic methods, particularly last observation carried forward (LOCF) compound, rather than resolve missing data issues. Intentionality is a hallmark of a quality randomized trial design. A useful analogy is in the construction of a building. All buildings, be they single family homes, chemical processing plants, or sports arenas, have the general property of keeping the outside out and the inside in. But an architect must have the building's intended use in mind, so that its entire structure conforms to those needs. The architects designing a clinical trial have the same need to go beyond describing the obvious intention to “compare treatments on an outcome, and spell out explicitly the primary research goal intended for the study. Is the primary goal to compare what would happen if everyone took drug A with what would happen if everyone took drug B? Or is the primary goal to compare the following two procedures: “offer everyone new drug A and if the patient refuses to take the drug or is nonresponsive on the drug, then give the standard drug, B” versus “offer only the standard drug, B”? Once these goals are established (or at least ordered in priority), then design choices follow. Such issues as what is randomized, what is fixed, what is allowed to vary outside of control, what is the analysis plan, can fall naturally once the research questions and goals are carefully articulated. The validity of the study design is defined by how closely it is likely to fulfill the primary goal. A study design, including the sampling strategy, assignment procedure, follow-up procedures, and analytical method, may be prone to systematically miss its goal, on average underestimating (or overestimating), the true but unknown quantity it is intended to evaluate. This is the design bias, alluding to the corresponding statistical concept of a biased estimator. For example, a trial that excluded cases that experienced a side effect would be expected to produce an overly optimistic estimate of an active drug's effect compared with placebo, if the side effect is negatively associated with the primary outcome of interest. Certain side effects may be positively related to outcome. For example, increased bioavailability of the drug may increase the beneficial effects of the drug but may also increase side effects. In this case, exclusion of cases that experienced a side effect would produce an overly pessimistic estimate of an active drug's effect compared to placebo. A study design may also tend to produce highly variable results, translating to a lack of confidence in their statistical stability, or a lack of precision. Such a situation occurs when the study involves too few subjects. The ultimate success of the completed trial depends on how closely it conforms to the intended design and how well the intended design addresses the primary goal, combining bias, and variability. Bias and precision of the design depend on the realities of study conduct in the real world. One of those realities is the phenomenon of “missing data,” the set of observations planned but not obtained. A design may be unbiased and precise in the fictional setting where all observations are complete but may fail in the real world where missing data occur. It follows that the designer must plan to accommodate incomplete observations. In this article we consider in particular the design consequences of anticipated patterns of missing data as well as mechanisms, as a prelude to a separate discussion of methods of analysis of such datasets (see companion paper by Siddique,4 page 793, in this issue). An array of statistical methods have addressed some of the problems resulting from missing data, and high quality methods are now implemented in many software packages. For modeling the type of repeated measures data that often occur in psychiatric trials, the Laird-Ware mixed effects model5 for conducting repeated measures analysis of variance, along with its direct and collateral descendents,6-8 provides a set of powerful tools for analyzing longitudinal data. The uptake of these new methods into psychiatric research was accelerated by publications9 that described their benefits and strengths. One much-cited and appreciated benefit is the easy way that they accommodate arbitrary patterns of missing data, at least in the sense that the software runs and produces apparently sensible estimates and inferential statistics (means, standard errors, confidence intervals, and tests of null hypotheses). Because of these convenient statistical packages, inferences can be drawn without requiring the researcher to think hard about the analysis of datasets with partially missing data on subjects, including truncation by dropout as well as sporadic gaps as patients miss visits. However, the fact that the software runs is not a guarantee that the output is sensible or valid. The validity of analytic results depends on the correspondence between the assumptions of the underlying model and the realities of the data, and to the sensitivity of those models to inevitable departures from the assumptions. Sometimes, a powerful analytic tool produces a “Type III error,” which is the correct answer to the wrong question.10 In keeping with this issue's emphasis on important matters of concern to the psychiatric researcher at the time of planning and study design, this article deals with the conceptualization of missing data from a design point of view. Good design may not eliminate the problem of missing data, but as we will show, it can reduce it, so that the modern analytic machinery can be used to extract statistical meaning from study data. Conversely, we note that when insufficient attention is paid to missing data at the design stage, it may lead to inferential problems that are impossible to resolve in the statistical analysis phase. The idea is summarized by the following “Method for Capturing Lions in New York City:” 1) by design, New York City does not harbor wild lions; and 2) the capture of tame lions is left as an exercise for the reader.