Discussion of reified Bayesian modelling and inference for physical systems by Michael Goldstein and Jonathan Rougier

We congratulate Goldstein and Rougier (GR) for a fine paper on the difficult and important subject of what can be learned from imperfect computer simulators of physical systems. That this assessment is difficult is widely recognized. The contribution 7 of GR is to propose a fully coherent framework for considering multiple simulators simultaneously. Our discussion is tripartite. First, we raise a point of clarification; second, we compare GR to current practice among climatemodellers; and third, we discuss 9 how GR can run into practical problems. Clarification: GR's formulation relies on a reified simulator f ∗ and an input vector (x∗,w∗) as described by Eq. (8). In this 11 formulation, (x∗,w∗) are system values. I.e., they are the true state of the real system we are trying to simulate. Our question is: “Do such values exist?” 13 In GR's example of the thermohaline circulation (THC), their reified simulator does not contain any further inputs or regressor functions than their generalized simulator f ′ (Eq. (29)). Thus f ∗ is a function of seven inputs: (T∗ 2, T ∗ 1 − T∗ 2, T∗ 3 − T∗ 1, ,K, q, T∗ 5); see 15 Table 1. GR's system is the Atlantic, though for simplicity they sometimes think of their system as the computer program CLIMBER17 2. With respect to these systems, T∗ 2, for example, is the temperature forcing of the North Atlantic. But neither the ocean nor CLIMBER-2 has a single T∗ 2 because neither system has a single atmospheric temperature over the entire North Atlantic. 19 Temperature varieswith location. Even in the simplifiedworld ofCLIMBER-2, temperature varieswith latitude. Similar comments apply to other inputs and to other systems that are more complicated than their reified simulators. As a general rule, systems 21 and their inputs are more complex than their simulators. So we ask GR to clarify what is meant by (x∗,w∗) both in general and in the example of the THC. 23 Contrast with current practice: In discussingGR, a useful point of comparisonwill be current practice among climatemodellers for assessing uncertainty. We begin by noting that climate modellers are aware of the desirability of constructing distributions 25 for uncertain parameters. A summary of some of their recent efforts can be found in Manning et al. (2004). There are several techniques in use; here we describe one used by Murphy et al. (2004) for the quantity called climate sensitivity which is, by Q2 27 definition, the equilibrium change in globally averaged surface temperature due to a doubling of atmospheric CO2. We use the symbol for climate sensitivity. 29 One result reported by Murphy is “a probability density function for the sensitivity of climate to a doubling of atmospheric carbon dioxide levels.” Murphy et al. obtain the probability density function (pdf) by sampling tuneable parameter values 31 uniformly over a range suggested by expert knowledge, running a climate simulator once for each draw of the parameters, weighting the simulations by how well they match current climate, extracting from each simulation, and then assembling the 33 weighted draws of into a posterior density. It is interesting to see how differently Murphy and GR use simulator runs. Murphy's simulator runs go directly into the 35 posterior distribution. GR's simulator runs go indirectly into the posterior; they go into improving the emulator for f by updating the posterior distributions for the quantities in the following equation: 37