Opening up scientific assessments for policy: The importance of transparency in expert judgements

Scientific assessments have become an important tool to support decision-making. Such assessments are formal efforts to assemble selected knowledge with a view toward making it publicly available in a form intended to be useful for decision-making. Important criteria for producing scientific assessments that are able to influence policy are salience, credibility, and legitimacy. We emphasize that for an assessment to be credible and legitimate, at least the expert judgements constituting the core of the assessment need to be made transparent. We propose a method to evaluate the quality of scientific assessments in that respect. This method is based on the evaluation by the PBL Netherlands Environmental Assessment Agency of a part of one of the most well-known scientific assessments, the 2007 Assessment Report of Working Group II of the Intergovernmental Panel on Climate Change (IPCC). Among the most common weaknesses found were insufficiently transparent expert judgements. We argue that authors and assessment practitioners should become more aware of the inevitable role of ‘expert judgement’, in which experts make an assessment despite high degrees of uncertainty, and should make those judgements more transparent, i.e. readers need to be able to follow the arguments of the assessment team. Furthermore, in order to become more reflective of different views, assessment methodology should incorporate a procedure of ‘open assessment’, for example by inviting ‘outsiders’ to participate in the quality control process.