Commentary: Divergent Thinking Tests Have Problems, But This Is Not the Solution

It is certainly true, as Silvia et al. (2008) write, that “after half a century of research, the evidence for global creative ability ought to be better” (p. 68). The authors believe—incorrectly, I think— that the reason that divergent thinking tests have not done a better job can be found in the various scoring systems that have been used when assessing divergent thinking ability. I have presented evidence elsewhere that creativity is not a general ability or set of traits or dispositions that can be applied across domains (Baer, 1991, 1993, 1994a, 1994b, 1998). In those studies, I used Amabile’s (1982, 1996) Consensual Assessment Technique (which is the basis for the subjective scoring technique proposed by Silvia et al. [2008]) to judge the creativity of a wide range of artifacts. What I found was that there is little correlation among the creativity ratings received by subjects across domains, and what little there is tends to disappear if an IQ test is also given and variance attributable to intelligence is first removed. If creativity is not a generic, all-purpose kind of skill—that is, if whatever it is that leads to creativity in, say, writing poetry does not also enhance creativity in teaching, creativity in cooking, and creativity in any other kind of activity—then we should not be surprised to find that tests of general creativity ability lack validity. In the arena of IQ testing, it has been shown that whatever it is that IQ tests measure is positively correlated with actual performance in a wide, domain-transcending range of tasks, but this is not the case for divergent-thinking testing. Perhaps the reason that “after half a century of research, the evidence for global creative ability ought to be better” (p. 68) is that, unlike intelligence, there simply is no general creativity skill to be measured. It may be that creativity is largely domain specific, a conclusion for which the evidence from the assessment of actual creative products is quite convincing. If so, then the construct of general creativity is a false one, and it doesn’t matter how you score creativity tests. Such tests can never be valid for the simple reason that they purport to measure something that doesn’t exist. But let me put that argument aside and assume, as Silvia et al. (2008) do, that the constructs of generic creative ability and generic divergent thinking ability reflect actual abilities that people possess in varying degrees. Given this assumption, it is fair to ask whether their proposed scoring represents a possible improvement on current methods of scoring. Unfortunately, Silvia et al. (2008) have failed to present convincing evidence that this might be the case. Due to space limitations, I will focus on just two problems with their method and analysis: a misunderstanding (and consequent misuse) of the Consensual Assessment Technique, and a deeply flawed validation process. There are other problems, such as the conceptual problem of confounding divergent thinking and evaluative thinking (because even if the new scoring system did lead to scores that correlated with actual creative performance, it might be caused solely by subjects’ evaluative thinking skills, not their divergent thinking skills—that is, it might be a test related to creativity, but not to divergent thinking ability).