Is What You See What You Really Get? Comparison of Scoring Techniques in the Assessment of Real-World Divergent Thinking

In recent years, the social sciences have seen a resurgence in the study of divergent thinking (DT) measures. However, many of these recent advances have focused on abstract, decontextualized DT tasks (e.g., list as many things as you can think of that have wheels). This study provides a new perspective by exploring the reliability and validity evidence for several methods for scoring real-world DT tasks (i.e., tasks situated within a real-world problem or situation). The results suggest a combination of objective and subjective scoring methods may be optimal for originality scoring for contextualized DT tasks, which stands in contrast to recent research suggesting the objective, percentage scoring technique may be optimal for scoring originality on abstract tasks.

[1]  H. Gough.  A Creative Personality Scale for the Adjective Check List , 1979 .

[2]  D. Hocevar,et al.  Ideational fluency as a confounding factor in the measurement of originality. , 1979 .

[3]  Jonathan A. Plucker,et al.  Is the Proof in the Pudding? Reanalyses of Torrance's (1958 to Present) Longitudinal Data , 1999 .

[4]  Mark A. Runco,et al.  Problem finding skills as components in the creative process , 1993 .

[5]  Paul J. Silvia,et al.  Creativity and Intelligence Revisited: A Latent Variable Analysis of Wallach and Kogan (1965) , 2008 .

[6]  James C. Kaufman,et al.  Comprar Essentials of Creativity Assessment | James C. Kaufman | 9780470137420 | Wiley , 2008 .

[7]  D. Hocevar,et al.  A Taxonomy and Critique of Measurements Used in the Study of Creativity , 1989 .

[8]  Gary A. Davis,et al.  Testing for Creative Potential , 1989 .

[9]  M. Runco,et al.  Creativity and the Finding and Solving of Real-World Problems , 1991 .

[10]  N. Milgram,et al.  Creative thinking and creative performance in Israeli students. , 1976, Journal of educational psychology.

[11]  T. M. Amabile.  Social psychology of creativity: A consensual assessment technique. , 1982 .

[12]  Mark A. Runco,et al.  Commentary: Divergent Thinking Is Not Synonymous With Creativity , 2008 .

[13]  P. Silvia,et al.  A snapshot of creativity: Evaluating a quick and simple method for assessing divergent thinking , 2009 .

[14]  J. Kaufman,et al.  Do experts and novices evaluate movies the same way , 2009 .

[15]  Mark A. Runco,et al.  Judgments of originality and appropriateness as predictors of creativity , 1993 .

[16]  M. Runco.  Children's divergent thinking and creative ideation , 1992 .

[17]  Russell Eisenman,et al.  Creativity, Originality, and Appropriateness: What Do Explicit Instructions Tell Us about Their Relationships?. , 2005 .

[18]  M. Runco,et al.  The Psychometric Properties of Four Systems for Scoring Divergent Thinking Tests , 1987 .

[19]  COMMENT ON “FLUENCY AS A PERVASIVE ELEMENT IN THE MEASUREMENT OF CREATIVITY” , 1971 .

[20]  R. Beghetto,et al.  Why Isn't Creativity More Important to Educational Psychologists? Potentials, Pitfalls, and Future Directions in Creativity Research , 2004 .

[21]  M. Runco.  Teachers' Judgments of Creativity and Social Validation of Divergent Thinking Tests , 1984 .

[22]  M. Runco.  Interrater Agreement on a Socially Valid Measure of Students' Creativity , 1987 .

[23]  D. Hocevar,et al.  Measurement of creativity: review and critique. , 1981, Journal of personality assessment.

[24]  H. Mirels,et al.  FLUENCY AS A PERVASIVE ELEMENT IN THE MEASUREMENT OF CREATIVITY1 , 1970 .

[25]  Jonathan A. Plucker,et al.  Is Originality in the Eye of the Beholder? Comparison of Scoring Techniques in the Assessment of Divergent Thinking , 2011 .

[26]  R. Sternberg,et al.  The Cambridge handbook of creativity. , 2010 .

[27]  D. Hocevar.  The unidimensional nature of creative thinking in fifth grade children. , 1979 .

[28]  J P GUILFORD,et al.  The measurement of individual differences in originality. , 1953, Psychological bulletin.

[29]  Jonathan A. Plucker,et al.  The death of creativity measurement has been greatly exaggerated: Current issues, recent advances, and future directions in creativity assessment , 1998 .

[30]  J. Plucker.  Student Responses to Creativity Checklists: Evidence of Content Generality. , 1996 .

[31]  Michael A. Wallach,et al.  Modes of thinking in young children: A study of the creativity-intelligence distinction. , 1965 .

[32]  Mark A. Runco,et al.  Scoring Divergent Thinking Tests Using Total Ideational Output and a Creativity Index , 1992 .

[33]  E. Torrance.  The nature of creativity as manifest in its testing. , 1988 .

[34]  William B. Michael,et al.  The Effects of Scoring Formulas on the Discriminant Validity of Tests of Divergent Thinking , 1979 .

[35]  W. Lyddon.  Creativity in Context: Update to "The Social Psychology of Creativity" , 1998 .

[36]  John Baer,et al.  A Comparison of Expert and Nonexpert Raters Using the Consensual Assessment Technique , 2008 .

[37]  Ronald E. Lesher.  Assessment of Creativity. , 1973 .

[38]  John Baer,et al.  Essentials of Creativity Assessment , 2008 .

[39]  W. R. Smith,et al.  Interpersonal and intrapersonal evaluations of creative ideas , 1992 .

[40]  S. Messick.  Validity of Psychological Assessment: Validation of Inferences from Persons' Responses and Performances as Scientific Inquiry into Score Meaning. Research Report RR-94-45. , 1994 .

[41]  Eunsook Hong,et al.  Creative thinking and creative performance in adolescents as predictors of creative attainments in adults: A follow‐up study after 18 years , 1993 .

[42]  John T. Willse,et al.  Assessing Creativity With Divergent Thinking Tasks: Exploring the Reliability and Validity of New Subjective Scoring Methods , 2008 .

[43]  D. Hocevar.  A comparison of statistical infrequency and subjective judgment as criteria in the measurement of originality. , 1979, Journal of personality assessment.

[44]  Nancy C. Canestaro,et al.  Creativity and achievement in design coursework , 1989 .

[45]  M. Runco,et al.  Problem discovery, divergent thinking, and the creative process , 1988, Journal of youth and adolescence.

[46]  M. Kane.  Terminology, Emphasis, and Utility in Validation , 2008 .

[47]  Mark A. Runco,et al.  The Reliability and Validity of Ideational Originality in the Divergent Thinking of Academically Gifted and Nongifted Children , 1985 .

[48]  N. Milgram,et al.  Quantity and Quality of Creative Thinking in Children and Adolescents. , 1978 .

[49]  E. Hong,et al.  Original thinking as a predictor of creative performance in young children , 1995 .

[50]  S. Messick.  Meaning and Values in Test Validation: The Science and Ethics of Assessment , 1989 .

[51]  Maud Besançon,et al.  Variability of creativity judgments , 2008 .

[52]  M. Runco.  Reliability and Convergent Validity of Ideational Flexibility as a Function of Academic Achievement , 1985 .

[53]  Paul J. Silvia,et al.  Subjective scoring of divergent thinking: Examining the reliability of unusual uses, instances, and consequences tasks. , 2011 .