Exchangeability and regression models

Sir David Cox’s statistical career and his lifelong interest in the theory and application of stochastic processes began with problems in the wool industry. The problem of drafting a strand of wool yarn to near uniform width is not an auspicious starting point, but an impressive array of temporal and spectral methods from stationary time series were brought to bear on the problem in Cox (1949). His ability to extract the fundamental from the mundane became evident in his discovery or construction of the eponymous Cox process in the counting of neps in a sample of wool yarn (Cox, 1955). Subsequent applications included hydrology and long-range dependence (Davison and Cox 1989; Cox 1991), models for rainfall (Cox and Isham 1988; Rodriguez-Iturbe, Cox and Isham 1987, 1988), and models for the spread of infectious diseases (Anderson, Cox and Hillier 1989). At some point in the late 1950s, the emphasis shifted to statistical models for dependence, the way in which a response variable depends on known explanatory variables or factors (Cox, 1958a). His contributions in both areas have been extraordinarily insightful, Cox processes being a fundamental class of point processes, and the Cox model playing a similar role in survival analysis. In addition to these, we have the Box-Cox transformation, binary regression models (Cox 1958b) and models relevant to agricultural field trials. This brief summary is a gross simplification of Sir David’s work, but it suits my purpose by way of introduction because the chief goal of this chapter is to explore the relation between exchangeability, a concept from stochastic processes, and regression models in which the observed process is modulated by a covariate. It is usual to introduce the notion of a stochastic process as a collection of random variables, Y1, Y2, . . ., usually an infinite set though not necessarily an ordered sequence. What this means is that U is an index set of statistical units, and for each finite subset S = {u1, . . . , un} of elements in U , the value Y (S) = ( Y (u1), . . . , Y (un) ) of the process on S has distribution PS on RS . This chapter emphasizes probability distributions rather than random variables. A real-valued process is thus a consistent assignment of probability distributions to observation spaces such that the distribution Pn on Rn is the marginal distribution of Pn+1 on Rn+1 under deletion of the relevant coordinate. A notation such as Rn that emphasizes the dimension of the observation space is not entirely satisfactory because two samples of equal size need not have the same distribution, so we write RS rather than Rn for the set of real-valued functions on the sampled units. A process is said to be exchangeable if each finite-dimensional distribution is symmetric, or invariant under coordinate permutation. The definition suggests that exchangeability can have no role in statistical models for dependence, in which the distributions are overtly non-exchangeable on account of differences in covariate values. I argue that