Relational and Arelational Confidence Intervals

In their recent article, Fidler, Thomason, Cumming, Finch, and Leeman (2004) noted that although researchers often use confidence intervals (CIs) in presenting data, they do not refer to CIs for drawing inferences. As the title of the article indicates, Fidler et al. consider this lack of reference to CIs in inference to be undesirable, even thoughtless. We offer an alternative interpretation. Unlike conventional tests, CIs, as commonly used, do not provide a direct comparison of groups or conditions. Consequently, researchers are acting thoughtfully by using CIs as descriptive indices rather than inferential ones. CIs are typically drawn around sample means. These CIs are arelational in that each provides information about only a single group or condition. There are many advantages to arelational CIs: They provide a rough guide to variability in data, a coarse view of the replicability of patterns, and a quick check of the heterogeneity of variance. Arelational CIs, however, do not reflect between-groups information and cannot be used for direct comparisons. A number of researchers have advocated the use of alternative CIs that do represent group or condition differences. For example, Tryon (2001) discussed how CIs can be constructed such that the presence or absence of overlap reflects hypothesistest outcomes. Masson and Loftus (2003) discussed CIs for main effects and interaction contrasts. Tukey (1977; see also Velleman & Hoaglin, 1981) recommended Tukey HSD-derived CIs for comparing means and medians in one-way designs. Thompson (2002) recommended reporting and drawing CIs around meta-analytic measures such as Cohen’s d or the proportion of variance explained. CIs in this class are termed relational, to describe their functionality. The advantage of relational CIs is that they may provide directly interpretable information about group or condition differences. The most salient disadvantage is the lack of standard convention for constructing and describing these intervals. To illustrate both types of CIs, we adapt here a well-known example from Hays (1994, p. 570, Table 13.21.2). The data are presented in Table 1, and condition means are plotted in the left panel of Figure 1. The dependent variable was the completion time for jigsaw puzzles. The independent variables were the overall shape of the puzzle (round vs. square) and the color scheme (monochromatic vs. colored). Each participant completed four puzzles, one in each condition. Repeated measures analysis of variance (ANOVA) reveals significant main effects of color scheme, F(1, 11) 5 13.9, p < .05, Cohen’s d 5 1.08, and shape, F(1, 11) 5 7.5, p < .05, Cohen’s d 5 0.79, but not a significant interaction, F(1, 11) 0, p 1, Cohen’s d 5 0. In Figure 1, for each bar in the plot we have drawn three different CIs as follows: Error Bar A is the 95% CI for the condition mean as calculated in Table 1. This CI includes variability across participants. In situations in which it is a nuisance, it may be removed by subtracting from each score the participant’s mean (Loftus & Masson, 1994; Masson & Loftus, 2003). Error Bar B is the 95% CI after this normalization; it does not reflect the overall variability across participants. Error Bar C is the 95% normalized CI from a pooled variance estimate. The disadvantage of Error Bar C is that heterogeneity of variance across groups is not graphically displayed. These three error bars are arelational; they do not reflect any particular comparison. Relational CIs can be constructed for specific contrasts, as